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ABSTRACT 

The National Longitudinal Transition Study (NLTS) of 
Special Education Students provides ongoing information regarding the 
transition of youth with disabilities from secondary school to early 
adulthood. The sample for the study, which began with a 1987 survey 
(Wave 1), involved more than 8,000 youth from the national population 
of secondary special education students. Wave 1 had four major 
components: the parent interview, abstraction of data from school 
records, the survey of secondary special education programs, and the 
nonresponse study. This report describes the study* s sample and study 
design features, including the definition of the universe; the 
methods used to build the sampling frame and select the sample; the 
response rates and efforts to increase them; procedures for weighting 
the data; and limitations of the sample and design. Conclusions 
address: (1) how well the sample is able to estimate school status; 
(2) comparison of the NLTS estimates of the number of students in 
each disability condition with U.S. Department of Education counts; 
and (3) limitations of the sample. The report examines the relatively 
high nonresponse rates but concludes that the NLTS meets high 
standards with respect to its procedures and will prove to be useful 
and valid for policy decisions and for social and scientific 
analyses. Sixteen tables detail the report's findings. (DB) 
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OVERVIEW 



This report documents the sample design and limitations of wave 1 of the National 
Longitudinal Transition Study (NLTS) of Special Educational Students. This overview 
section describes the role of this report in the NLTS methodology report series, 
describes the purposes of the NLTS, summarizes the major NLTS survey components 
and the types of information obtained from each component, and provides a timeline of 
the major sampling and data collection milestones. 

Methodology Reports 

This is the second in a series of three methodology reports documenting procedures 
used to obtain and analyze nationally representative data concerning the transition of 
secondary school special education students to early adulthood. These reports include: 

• The Report on Procedures for the First Wave of Data Collection (1987) 
describes the data collection instruments and procedures (Wagner, Newman, 
and Shaver. 1989). 

• This report describes the procedures used to define the sampling frame, select 
the sample, and compute sample weights. 

• The third report (in preparation) deals with the formation, reliability, and validity 
of variables used in tabulations and other analyses. 

Study Purpose 

The NLTS was mandated by the U.S. Congress in 1983 to provide information to 
practitioners, policymakers, researchers, and others in the special education community 
regarding the transition of youth with disabilities from secondary school (grade 7 or 
above) to early adulthood. In particular, Congress was interested in taking stock of how 
youth with disabilities were faring under the Education of the Handicapped Act (EHA) in 
terms of receiving education and related services that would help them make a 
successful transition from school to employment and independent living. The Office of 
Special Education Programs (OSEP) of the U.S. Department of Education contracted 
with SRI International to determine a design, develop and field test data collection 
instruments, and select a sample of students for a study that would meet the 
congressional mandate. In April 1 987, after completing the study design, SRI began the 
actual study. 



In the field of research on youth with disabilities, the NLTS is unique in severa! 
respects. The NLTS sample is large, permitting us to estimate with fairly high precision 
many of the characteristics of youth with disabilities and their experiences in 
adolescence and early adulthood. The data were collected in 1987 for a nationally 
representative sample of more than 8,000 youth who were secondary special education 
students in the 1985-86 school year. It is also nationally representative of youth in each 
of the 1 1 federal disability categories. Therefore, for the first time we know on a 
nationwide basis what the transition experiences were for youth with mental retardation, 
for example, and how they differed from those of youth with orthopedic impairments or 
multiple handicaps. 

The NLTS is also unusual in its longitudinal design. The students for whom data 
were gathered in 1987 are being tracked through the years of the study so that follow-up 
data can be collected about them late in 1990. These follow-up data will enable the 
estimation of trends in axperiences as youth age. For example, we will be able to 
describe the movement in and out of jobs and in and out of school that often 
characterizes youth in these •^floundering years.** Finally, the NLTS is extremely broad in 
scope, gathering information on a wide range of characteristics, experiences, and 
outcomes of youth with disabilities, including the following: 

Individual characteristics (e.g., disability-related characteristics and severity). 
Family characteristics (e.g., demographics). 

• Parent expectations for youth in the areas of education, employment, and 
independence. 

• Social experiences (e.g., belonging to school or community groups, socializing 
with friends). 

• School programs (e.g., courses taken, related services provided, educational 
placements). 

• School characteristics and policies (e.g., type of school attended, policies 
related to mainstreaming, programs available for special education students). 

• School achievement and completion (e.g., grades received, absenteeism, 
dropout/graduation behaviors). 

• Services provided by the school and other sources (e.g., job training, physical 
therapy, counseling). 

• Employment characteristics (e.g., rates of employment, job types and duration, 
wages). 

• Independent functioning (e.g., residential independence, financial 
independence, functional abilities). 

• Postsecondary education participation in vocational schools and 2-year and 4- 
year colleges. 
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The study addresses both descriptive and explanatory research questions. The 
focus of description is to understand better the patterns of experiences of youth in school 
and in transition to adulthood. The primary explanatory research questions involve 
identifying factors that are associated with the effective transition of youth with 
disabilities from secondary school to employment, further training and education, and 
independent living. Of particular interest is identifying what schools do in the way of 
programming, staffing, and organization that is associated with successful transition. 

To document several aspects of the study methods, the NLTS is producing a series 
of three reports that describe (1) the sample and the study's design limitations, (2) data 
collection procedures, and (3) measurement and analysis approaches. This report 
documents the sample design used to develop the primary database* for the NLTS, 
including the definition of the universe; the methods used to build the sampling frame 
and select the sample; the response rates and efforts to increase them; procedures for 
weighting the data; and limitations of the sample. Knowing how the sample was 
generated and weighted, and limitations of the sample (caused primarily by different 
types of nonresponse, as explained later), provides a context for understanding the 
nature and meaning of the findings the study generates. Documenting sampling 
procedures from a study that is this large and complex can also provide useful guidance 
to other researchers who will be working in the transition arena in the future. 

Study Components 

NLTS data collection and analyses are guided by a conceptual framework (shown in 
Figure 1) that specifies the main categories of factors expected to influence transition 
outcomes. Several sources of information were used to obtain the data needed to 
elaborate the categories in the framework. For example, data regarding sciiool and 
district factors and school programs were best collected from school personnel and 
records, while family characteristics were best reported by parents of youth in the 
sample. 



The "primary database" refers to data intended to be collected for the full sample and is the database 
used for most analyses. Additional data are being collected for selected subsamples of youth; the 
designs of those substudies are documented in separate reports about the substudies. 
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Four data collection components were designed to obtain the data specified by the 
conceptual framework: 



The parent Interview. The parents/guardians of sample youth were 
administered a structured interview by telephone in the summer and fall of 
1987 to obtain information on youths' individual and family characteristics; 
services received; social integration; outcomes in the areas of employment, 
educational attainment (including postsecondary education), and 
independence; and parent expectations of future achievements. This survey 
will be repeated in 1990, when youth will be interviewed if they are able to 
respond. 

• Abstraction of data from school records. School or district staff were 
recruited in the 1987-1988 school year to abstract data from school records for 
the most recent year youth were in secondary school (either 1986-87 or 1985- 
86) and to record the data on forms provided by SRI. School record data 
concerned the disabilities for which youth received special services, grade level 
in school, educational setting, courses taken, grades received (if irr a grpded 
program), related services provided, attendance, IQ, mihirriuffl compet -^y test 
experiences, and end-of-year status (e.g., dropped out, graduated, promoted to 
next grade level). School transcripts will be collected in 1990-91 for youth who 
had been in secondary school at any time since the 1986-87 school year. 

• The Survey of Secondary Special Education Programs. A mail 
questionnaire was sent to the principals of the secondary school most recently 
attended by each youth. The first part of the questionnaire related to general 
characteristics of the school and its student body and was usually completed by 
the principal. Later sections of the questionnaire related to specific kinds of 
services and instruction provided to secondary special education students (e.g., 
life skills training, job skills training) and were usually completed by special 
education personnel. The later sections also addressed community resources 
for the disabled. 

The Nonresponse Survey. In survey research, there is always a concern that 
respondents to a survey differ systematically in some way from 
nonrespondents, thereby introducing bias into the survey data. To determine 
whether bias existed in the parent/guardian telephone interview data, in-person 
interviews were conducted with a sample of parents/guardians who had not 
been reached by telephone. These parents/guardians were administered a 
somewhat simplified version of the parent/guardian questionnaire. By 
comparing the sample of nonrespondents with parents/guardians who had 
responded to the telephone survey, bias in the sample was identified and 
partially adjusted for. 

Further information about the content of these questionnaires and the procedures 
used to collect the data (including procedures for encouraging response) is available in 
the NLTS Report on Procedures for the First Wave of Data Collection (1987) (Wagner, 
Newman, and Shaver, 1989). 
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NLTS Wave 1 Sampling and Data Collection Milestones 

The next few sections discuss the procedures used to select the NLTS sample, 
obtain the data, and weight them. Critical NLTS wave 1 sampling and data collection 
activities and their associated dates were as follows: 

• Select the initial sample of 450 local education agencies (LEAs) (10/85). 

• Recruit initial LEA sample to participate in the study (10/85 to 6/86), 

• Select sample of state-operated schools for the deaf and blind (12/85). 

• Recruit state schools to participate in the study (12/85 to 5/86), 

• Select supplemental sample of LEAs to replace those refusing to participate 
(2/86). 

• Recruit replacement LEAs to participate in the study (2/86 to 6/86), 

• Select students from rosters of participating LEAs and state-operated schools 
and obtain student addresses and telephone numbers (10/85 to 9/86). 

• Solicit written parental consent for sampled students to participate in the study 
(5/86 to 10/86), 

• Notify LEAs, state-operated schools, and parents that the NLTS study is 
beginning (5/87), 

Main wave of telephone interviewing of parents (6/87 to 9/87), 

• Follow-up telephone interviewing of parents (9/87 to 1 1/87), 

• Identify schools attended by sampled students (1 0/87 to 1 2/87). 

• In-person interviewing of nonrespondents to the telephone interview (1 1/87 to 
2/88). 

• Recruit data abstractors in schools attended by sampled students, abstract 
school records, and administer secondary school program survey (1/88 to 
7/88). 

• Analysis of nonrespondent bias and adjustment of sample weights (5/88 to 
8/88). 
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THE LEA SAMPLE 



The focus of the NLTS is to make reasonably precise estimates about what 
happened to special education students (overall and by disability category as defined by 
the federal government*) in the context of their local environment, which includes the 
lOv-^al education agency (LEA) and other community resources. In selecting the sample, 
the LEA served as the vehicle to obtain a sample of special education students; it is the 
primary sampling unit. The special education student constitutes the secondary or final 
sampling unit. This section describes the procedures used to define the universe of 
LEAs, create the sampling frame, stratify the LEAs in the sampling frame, select LEAs 
for inclusion in the sample, obtain LEA rosters of secondary special education students, 
and assess the amount of nonresponse bias associated with the LEA participation rate. 



Defining the Universe of LEAs 

The first step in defining the universe of LEAs involved conferring with OSEP and the 
National Center for Education Statistics (NCES). The universe was initially defined as all 
operating LEAs offering grade 7 or higher. Nonoperating LEAs that did not administer 
any schools were eliminated from consideration,** as were various categories of local 
and state educational "districts" that appear on standard listings of educational 
institutions, including (1) vocational-technical districts (except ones that operated as 
regular LEAs); (2) supervisory unions, area educational agencies, interim districts, 
boards of county education services, or other superordinate units; (3) public agencies, 
such as state education agencies (with the exception of the Department of Education in 
Hawaii, which is an LEA); (4) Bureau of Indian Affairs schools; (5) achievement centers 
and regional resource centers; (6) private agencies, such as homes for delinquent youth; 
(7) Texas Independent State School Districts, which primarily are correctional facilities 
and homes for delinquent youth; and (8) LEAs from Puerto Rico, Guam, and other 
territories. Because of the focus on special education students in secondary schools, 
districts that served only students in kindergarten through sixth grades were also 
excluded. Districts that served 10 or fewer students were excluded because, on 
average, approximately 10% of all students are in special education, and so these 
districts would not be expected to have any special education students. 



The 1 1 handicapping conditions are learning disabled, mentally retarded, <^eriously emotionally 
disturbed, orthopedically impaired, speech impaired, deaf, hard of hearing, visually impaired/blind, 
deaf/blind, other health impaired, and multiply handicapped. 

According to NCES, a nonoperating system is a self-contained local public school system having its 
own decision-making board of control that does not operate schools, but pays tuition to other operating 
systems for the education of the children living within its boundaries. 
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Creating the Sampling Frame 



To create a sampling frame or master list of LEAs, a composite database was 
developed from several available sources, including the most current data from the 
public school universe file maintained by Quality Education Data (QED),* and the 
National Center for Education Statistics (NCES) tapes for 1980-81 and 1981-82. When 
necessary, these data were supplemented with telephone calls to individual states and 
districts. 

Using these sources, a master list of LEAs was created, using the following 
procedures: 

• Obvious errors on each tape were eliminated, such as blank or duplicate 
records, no names, spelling errors, invalid codes, and extreme outliers. This 
was accomplished by comparing data on all tapes and, when necessary, calling 
states and LEAs for information. 

• All nonoperating LEAs, supervisory unions, vocational-technical districts, and 
relevant public agencies were removed (see previous discussion). 

• Using QED (1 983) and NCES (1 980-81 ) data, an LEA was included on the 
master list if (1) the district identification number was on both sources or (2) the 
number was on at least one source and was verified by the later NCES (1 981 - 
82) tape and a telephone call. 

• When an LEA was not listed by all sources, checks were made for incorrect or 
changed district names and altered district identification numbers. When this 
possibility was taken into account, we found that many districts were on all 
three sources but were listed under different names or identification numbers. 
Data were then merged into a single record. In general, the QED identification 
number and/or district name was used when sources disagreed because the 
names and addresses of school officials from QED were to be used to obtain 
permission to participate in the longitudinal study. 

• QED had artificially merged some K-8 and 9-12 LEAs for mailing purposes. In 
these caf^es. NCES 1981-82 data were used, which kept them separate. 

• Checks were made for LEAs that had the same identification number on 
different tapes but had different names. Most were a result of different 
spellings or misspellings. When there were discrepancies, the most 
appropriate name was selected (usually QED's), and LEAs were separated and 
merged as appropriate. 

To test the accuracy of the initial master list of secondary LEAs, it was matched with 
lists of LEAs submitted by seven states directly to SRI. There was a 9975% agreement 
between the two lists. 



The 1 983 Quality Education Data, Inc., (QED) database was used both to construct the sampling frame 
for school districts and to conduct an analysis of the differences between participating and 
nonparticipating districts. QED Is a private nonprofit firm located in Denver, Colorado. 
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In this way, an initia! master list was developed, containing 13,975 secondary-level 
LEAs that were expected to have at least one special education student. This list was 
later revised on the basis of an SRI survey of 1 ,600 of these LEAs (described later) to 
reflect the fact that approximately 57% of districts no longer offered secondary-level 
instruction or had merged with other LEAs. The revised universe contained an 
estimated 13,180 LEAs.* 



Stratifying the Universe 

To increase the precision of estimates that would have resulted from a simple 
random sample, and to increase the representativeness of the sample, the LEA universe 
was stratified by region, district size (enrollment), and a measure of district wealth. 
These variables, their sources, and the rationale for their selection are described below. 

flefif/o/>— The region variable captures essential political differences, as well as 
subtle differences in the organization of schools, the economic conditions under which 
they operate, and the character of public concerns. For example, at the time that the 
LEAs were selected, the "sunbelt" Southern states had growing enrollments together 
with relatively healthy economies that were able to support various endeavors to 
improve education, wliereas the declining industrial states of the Northeast faced 
decreases in the resources that could be used for school improvement. 

For this study, the regional classification variable used by the Department of 
Commerce, the Bureau of Economic Analysis, and the National Assessment of 
Educational Progress was adopted: 

Northeast Connecticut, Delaware, District of Columbia, Maine, Maryland, 
Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode 
Island, and Vermont. 

Southeast. Alabama, Arkansas, Florida. Georgia, Kentucky, Louisiana, 
Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West 
Virginia. 

• Central. Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, 
Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin. 

West/SouthwQSt Alaska, Arizona, California, Colorado, Hawaii, Idaho, 
Montana, Nevada, New Mexico, Oklahoma. Oregon, Texas, Washington, 
Wyoming, and Utah. 



This total does not include the unknown number of LEAs that did not offer secondary-level instruction 
when the original list was compiled and subsequently did offer such instruction, or the unknown number 
of LEAs that disassociated from larger LEAs after the original list was compiled. 
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District Size (Enrollment)— LEAs vary considerably by size (the most useful 
available measure of which is pupil enrollment). A host of organizational and contextual 
variables with considerable potential influence over the operations and effects of special 
education and related programs are associated with size. These include the extent of 
district administrative/supportive capacity, the degree of specialization in administrative 
structure, the nature of citizen and interest group activity in education, and the 
characteristic relationship with state and federal governance systems. In addition, total 
enrollment served as an initial proxy for the number of special education students served 
by a district. 

The district size stratification variable was initially a nine-category scale, the first 
seven of which corresponded to the scale used in NCES research. After examining the 
number of districts and proportions of students accounted for by each of these nine 
categories and various other demographics of the LEAs, categories were merged into 
six enrollment strata. These were defined to be consistent with other research and to 
ensure that each stratum of LEAs was homogeneous, yet more or less distinct from 
other strata. The principal strata were: 

• Huge districts (enrollment of 50,000 or greater). These were typically districts 
in large urban centers. This stratum contained fewer than 1% of LEAs but 
slightly more than 17% of secondary students. 

• Very large districts (enrollment between 25,000 and 49,999). These were 
typically districts in medium to large cities or large county systems, which often 
were organizationally complex and very likely to be divided into subdistricts. 
Very large LEAs constituted fewer than 1% of districts but almost 9% of 
students. 

• Large districts (enrollment between 10,000 and 24.999). These were districts 
in small to medium-sized cities or large county systems. They were also 
organizationally complex, but these systems tended to be centralized. Large 
districts constituted 3-5% of LEAs and served approximately 18% of secondary 
students. 

• Medium-sized districts (enrollment between 2,500 and 9,999). Typically, 
these were suburban districts, districts in large rural towns, or small county 
systems. Medium-sized -iistricts constituted 22% of LEAs and served 
approximately 35% of secondary students. 

• Small districts (enrollment between 500 and 2,499) . Most were rural districts. 
This stratum accounted for 38.5% of LEAs and served 1 7% of secondary 
students. 

• Very small districts (enrollment fewer than 500), These were small rural 
districts. This stratum accounted for 347o of LEAs but served only 3% of 
secondary school students. 
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An additional logistical reason for stratifying on district size was to reduce the 
number of LEAs that would need to be sampled to achieve adequate representation 
among larger districts. For example, without stratification, we would expect, out of a 
sample of 500 LEAs, to select fewer than 25 huge, very large, or large districts, even 
though these strata account for 44% of all students. 

District/Community Wealth— The ratio of students receiving Title I funds to the total 
student population in an LEA was used as a measure of district/community wealth and 
as an initial proxy for the amount of services available in the community surrounding the 
LEA. This index estimated the percentage of youth in an LEA who were below the 
poverty level. It was hypothesized that this index would relate to the types of programs 
offered in schools and to the services likely to be available in a community. Four 
categories of this variable were created for sampling purposes: 

• High (0% to 4% disadvantaged youth). It was hypothesized that LEAs with 
very small percentages of low-income youth would, in general, be least 
influenced by these youth when making decisions about programs. As 
relatively high-wealth districts, one would expect them to have a relatively large 
number of services available in the surrounding community. 

• Medium (5% to 9%). It was hypothesized that LEAs with 5% to 9% low-income 
students would be more likely than wealthy districts to take these youth into 
account when making decisions about programs. The overall impact on 
programs, however, might be limited. We expected, on the average, fewer 
community services to be available in these LEAs than in LEAs in the low and 
very low categories. 

• Low (1 0% to 1 9%). LEAs with 1 0% to 1 9% low-income youth would be more 
likely to take these youth into full account when making decisions about 
programs. One would expect a greater awareness of the problems of these 
youth (and possibly of special education students if the disadvantaged and 
special education programs overlapped). In some cases, even fewer 
community services would be available than in wealthier districts. However, as 
the district and the community received federal and state funds to respond to 
the larger disadvantaged population, more services might be available. 

• Very low (20% and over). It was assumed that with large numbers of 
disadvantaged youth, the entire academic program would be affected. As the 
percentage increased, programs for disadvantaged youth would dominate the 
curriculum. The services available would depend greatly on the amount of 
federal and state assistance received. 

The three variables generated a 96-cell grid into which the entire universe of LEAs 
was distributed. 



Selecting LEAs for Inclusion in the Sample 



To obtain additional information on a larger number of districts than would eventually 
be included in the LEA sample, SRI randomly selected a first-stage stratified sample of 
1 ,600 LEAs (approximately 11% of the secondary LEA population) by allocating the 
1 ,600 LEAs among size strata so that (1) ail huge and very large districts were chosen, 
(2) a majority of the large districts, with enrollments between 10,000 and 25,000, were 
included; (3) the remaining sampled LEAs were distributed among the other strata in 
approximate proportion to the number of students contained in each. The net effect of 
these decisions was to ensure the selection of a very high proportion of larger districts 
and a lower proportion of smaller districts. Weighting procedures (described later in this 
report) were used to adjust estimates for unequal LEA (and student) selection 
probabilities and for nonresponse bias. 

At SRI's request, Chilton Research Services (CRS) conducted a survey of the 1,600 
LEAs in the first-stage sample to determine more precisely the services available to 
special education students and to estimate the number of special education students 
served by each district. The survey revealed that 47 LEAs classified by NCES as 
offering secondary school instruction no longer did so. In addition, 4 LEAs had merged 
with other districts and were no longer distinct entities. Subtracting these 51 LEAs from 
the first state sample of 1 ,600 left a sample of 1 ,549 districts. Of these, 1 ,45U 
responded and 99 either refused to participate or were not reached, yielding a response 
rate for secondary-level LEAs of 93.6%. 

The estimated number of LEAs offering grade 7 or higher in the universe was revised 
by eliminating the proportion of LEAs in each stratum found not to have grade 7 or higher.* 
The result was a revised estimated universe of 13,180 LEAs (reduced from 13,975). 

A second-stage sample of 450 LEAs was selected from the 1 ,450 respondents to the 
first-stage survey, using stratified random sampling procedures. The number of selected 
LEAs in each stratum was approximately proportional to the number of LEAs in the first- 
stage sample. The procedures used to contact these LEAs are described in the next 
section. As described there, the response rate among the 450 LEAs was not as high as 
desired, so the second-stage sample was augmented with an additional sample of 178 
LEAs selected from the same sampling strata as refusals from the original set of 450. In 
total, 303 LEAs participated in the NLTS.** 



For example, if in a stratum, 2 of 10 sample LEAs were found no longer to teach secondary students, 
and the original universe listed 50 LEAs in that stratum, we multiplied the proportion (2/10) by the 
stratum population (50), and subtracted this number from the original universe number. In this example, 
the revised estimated number in the universe for this stratum would be 50 - [(2/10) (50)] - 40. The 
estimate does not include an adjustment for nonresponse. 
** In addition, as described later, 84 state-operated schools for the deaf and visually impaired were added 
to the sample, and 22 of these schools participated in the NLTS. 
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Table 1 displays the revised universe of secondary school districts and the number 
of secondary school districts that participated in the survey by region, enrollment size, 
and district wealth. 



Obtaining LEA Rosters of Secondary Special Education Students 

In the summer of 1985, the U.S. Department of Education sent letters to the state 
special education directors of all 50 states describing the NLTS and requesting permission 
to contact sampled school districts about participation in it. Every state granted approval 
to contact districts. In October 1985, SRI sent a letter about the study to the 
superintendents of 450 districts that were selected in the original sampling process. The 
letter briefly described the purpose of the study, invited the district to participate, and 
requested that district personnel provide a roster of all special education students in 
grades 7-12 who were also ages 13-21 (e.g., excluding 12-year-old 7th-grade students) in 
the district at that time. The instructions to districts asked that they include in their rosters 
special education students for whom the district is legally or financially responsible, even if 
those students did not attend one of the district's schools (e.g., they attended a private day 
school, a state-operated school for the deaf or blind, a neighboring schoo,- district, etc.).* 
Information requested of the districts included the name of the student, the type of 
disability (using the 1 1 federal categories), the birth date or age of the student, the grade 
level of the student, and the name of the school or agency that the student attended. 
District staff were informed that SRI would be using the rosters to select students 
randomly to be in the study and that we would be contacting the parents of selected 
students to request their voluntary participation, in the mailing was a copy of a letter from 
the U.S. Department of Education, Family Educational Rights and Privacy Act (FERPA) 
office, stating that SRI was an authorized representative of the U.S. Department of 
Education and that, for the purpose of this national evaluation of a federally sponsored 
program, education agencies could release student information to SRI without violating 
FERPA. Also enclosed were instructions about providing roster information and a roster 
cover sheet. (Roster information could be provided on a computer tape or printout; an 
existing document, such as a class list, that contained the necessary information on each 
special education student; or a hand- or typewritten roster on a form SRI provided). 
Districts that did not respond within the first 2 weeks were sent a reminder letter. Two 
weeks following the second letter, project staff called staff in districts that had not yet 
responded to our request to explain the study, answer questions about it, discuss how 
student rosters could be provided, and generally encourage participation in the study. 



Districts followed a variety of policies concerning the listing of such students, and some districts may 
have omitted them from their lists. In addition, some severely disabled special education students may 
never have been in LEAs. Thus, the NLTS sample of students from LEA rosters is likely to exclude 
some of the most severely disabled students. A supplemental sample of special education students 
from state-operated schools for the deaf and blind (discussed later) increased the representation of 
severely disabled students in some handicapping conditions (i.e., deaf, hard of hearing, visually 
impaired, blind, deaf/blind, and, to a lesser extent, multiply handicapped). 
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Most responding districts provided SRI with printouts or SRI forms that contained the 
necessary information on each special education student. Two categories of exceptions 
were as follows: 

Four very large or huge districts with computer capabilities requested that they 
select the sample rather than send us hundreds of pages of roster information. 
These districts first provided counts of special education students by disability 
category and by student age (13-15, 17-18, and 19-21). From these counts, 
SRI determined the number of youth in each sampling category to select and 
provided detailed instructions on how to select the sample. Families who 
wished to participate returned completed consent forms to SRI. 

District staff from 28 other LEAs who were concerned about violating student 
confidentiality rights by giving us student names were informed that, rather 
than release the names of students, they could provide a roster with an 
identification number or initials for each student. In these cases, SRI staff 
selected the sample and informed the district of the ID numbers or initials of 
the students selected. 

Special education students sampled from these districts accounted for 18.8% of all 
sampled students. Districts following either of these two sampling procedures sent 
parent consent forms prepared by SRI to the families. Families who wished to 
participate returned completed consent forms to SRI. Only 30% of selected families 
chose to complete the consent forms. SRI was not able to contact families that did not 
return the consent form to pursue follow-up activities, resulting in a much lower 
participation rate from families in these districts than from families in other districts. 

Refusals to participate by the initial sample of 450 districts were much higher than 
anticipated on the basis of the survey of 1,600 districts described previously. We had 
hoped to solicit 400 participating districts from the sample of 450. Several months after 
the roster and student sampling processes began. 132 districts had refused to participate. 
Project staff determined that a replacement sample of districts would be needed to 
generate a sample of students that was representative of students nationwide in each 
disability category and that was sufficient for analysis purposes. Another 178 districts 
were selected that were matched to refusing districts on each of the stratification variables 
in the district sampling frame. Initial contacts to these oistricts began in January 1986, 
using the same follow-up procedures for this group as for the first group of districts. 

Many youth in the deaf and visually impaired categories attend state-operated 
schools for the deaf and blind and did not appear on the rosters of sampled districts. To 
obtain the needed representative sample of youth in these categories, we defined an 
additional stratum consisting of all 84 state-operated schools for the deaf and blind, and 
selected all schools in this stratum. This expanded the universe to include all LEAs and 
state-operated schools for the deaf and blind serving secondary special education 
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students.* We used the same contact and follow-up procedures for state-operated 
schools as for the first group of districts. 

Contacts to districts and state-operated schools about obtaining student roster 
information continued through June 1986. When the student sample was selected from 
the roster, we sent a list of the selected students and requested that the district or state - 
operated school provide parent names, addresses, and home telephone numbers so 
that we could contact the families to request their consent to participate in the study. 
Most districts that would not provide the students' names agreed to send the parents 
material provided by SRI to elicit participation. This material included a letter explaining 
the purpose of the study and briefly how the youth were selected, a form requesting the 
parent's name, address, and telephone number, and a postage-paid envelope. A 
handful of districts dropped out of the study at this point because they did not provide 
parent names, nor would they agree to send the parent materials themselves. 

Table 2 summarizes the results of efforts to gain rosters of special education 
students from districts and special schools. Of the 628 sampled districts and 84 
sampled state-operated schools, 303 districts and 22 state-operated schools supplied 
rosters. Eleven state-operated schools and 174 districts refused to participate. 
Concerns of staff burden in providing student rosters and other information for the study 
and co.icerns about student privacy were the two most commonly cited reasons for 
refusals. Staff in the remaining districts and state-operated schools agreed to participate 
and said they would provide a roster, but despite numerous reminder telephone calls 
over many months, the rosters were never sent. Thus, approval of the study did not 
always mean participation in the study. 



Table 2 

RESULTS OF REQUESTS FOR STUDENT ROSTERS 



Sample Type 


Sampled 


Provided 
Roster^ 


Direct 
Refusal 


Indirect 
Refusal^ 


Participation 
Ratec 


Original LEAs 


450 


226 


132 


92 


50.2% 


Replacement LEAs 


178 


77 


42 


59 


43.3 


Total LEAs 


628 


303 


174 


151 


48.2 


State-operated schools 


84 


22 


11 


51 


26.2 



^ Includes districts and special schools that sampled students themselves without actually providing a roster 
to SRI. 

^ Indirect refusals include districts and special schools that agreed to participate but did not provide rosters. 
^ Participation rate shown in this table excludes first-stage LEA refusals. Overall participation rate for 

combined original and replacement LEAs including first-stage refusals was 45.2%. Overall participation 

rate for special schools was 26.2%. 



* Although the supplemental sample was selected to obtain youth in the deaf and visually impaired 
categoriiss, a small number of youth in the deaf/blind, multiply handicapped, and hard of hearing 
categories were also attending the state-operated schools and were included in the sample. 
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Assessing Potential Nonresponse Bias in the LEA Sample 

Given the fairly low response rate of districts and state-operated schools, SRI was 
concerned that bias would be introduced into the sample if nonparticipating districts were 
systematically different from participating districts in ways that could affect the topics 
under study. The amount of bias was estimated by comparing telephone survey 
responses of the 1 ,450 first-stage sample respondents and demographic data on these 
districts available from NCES and QED with data for the 303 respondents to the second- 
stage sample of LEAs (for whom nonresponse bias is a concern). 

The following variables from the telephone survey of the first-stage sample were 
used to examine bias: 

• Disabilities served. Whether the LEA served learning disabled, mentally 
retarded, speech impaired, orthopedically impaired, emotionally disturbed, hard 
of hearing, deaf, visually impaired, deaf and blind, multiply handicapped, or 
other health impaired students. 

• Number of special education students. Whether the LEA served fewer than 
10, 11 to 50, 51 to 100, 101 to 500, or more than 500 special education 
students. (Approximately 1% of the LEAs could not provide this information.) 

• Vocational rehabilitation (VR) participation. Whether the LEA had few, 
some, or most of its special education students participating in VR. 

• School-based resources. An index of the amount of school-based resources 
available for special education students, defined on the basis of the availability 
of school vocational programs and placement personnel, variety of settings for 
serving learning disabled and mildly mentally retarded students, and existence 
of transitional programs and occupational and physical therapy services. 

Community resources. An index of the amount of non-school-based 
resources available for special education students, based on the proximity and 
quality of community services and employment opportunities for youth with 
disabilities. 

• LEA configuration. Whether the LEA's special education students were 
served by (1) a city office or special education district (i.e., an "up" 
configuration), (2) neighboring school districts (i.e., an "across" configuration), 
(3) state-operated schools, private day-care or residential schools, 
developmental disability institutes, mental health institutes, or other 
organizations serving special education students (i.e., an "out" configuration), 
or (4) entirely by the LEA itself. Note that the mutually exclusive categories are 
(1) "up (only)," (2) "across (only)," (3) "up and across," (4) "out," and (5) "in 
house." 
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• Metropolitan status. Whether the LEA was in an urban, suburban, or rural 
area. 

• Percentage minority students In LEA. Percentage of the total student 
population in the LEA that was minority. 

• Grades served. Whether the LEA served grades kindergarten through 8. 
kindergarten through 12, or secondary students only. 

• Age limit on services. Whether the state-determined age limit on services 
was 18 or an age greater than 18. 

Table 3 compares the population values as estimated using the 1,450 respondents 
to the first-stage sample of LEAs and the 303 respondents to the second-stage sample 
of LEAs. All estimates are weighted to the universe of 13,180 LEAs. Three measures of 
bias are computed: 

• The absolute bias, computed as the absolute value of the difference between 
the estimate of the population value derived from the first-stage sample 
(denoted Xp) and the estimate of the population value derived from the 
second-stage sample (denoted Xe). 

• The relative bias, computed as the absolute bias divided by Xp. 

• The Z-value of the bias, defined as the absolute bias divided by the standard 
deviation of the bias. In computing the standard deviation, we used the fact 
that the second-stage sample is a random subsample of the first-stage sample 
(within the sampling strata), and therefore the first-stage sample can be 
regarded as a known "universe" from which a sample was taken. The formula 
for the variance of the bias is as follows: 



where N = 1 3.1 80 is the total number of LEAs in the universe, k is an index for 
strata, there are K = 96 strata. Mk is the number of first-stage respondent LEAs 
in the Wh stratum, mk is the number of second-stage respondent LEAs in the 
/cth stratum, Nk is the number of LEAs in the universe in the Mh stratum, and 
Vk is the variance of the responses of the Mk first-stage respondents in the Wh 
stratum. 
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In examining Table 3, we looked for variables for which all three bias indicators were 
large— an absolute bias of 0.05 or larger (for the variables whose responses are 
percentages), a relative bias of 0.15 or larger, and a Z value in excess of 1 .96 (because 
the Z value is from a two-sided test, this is equivalent to requiring differences to be 
significant with 95% confidence). Relevant observations from Table 3 are as follows: 

• The disabilities served by LEAs were accurately represented. The largest 
discrepancy was in percentage of LEAs serving orthopedically impaired 
students; there, the relative bias was only 8.6%. 

• The number of special education students served by the LEAs was accurately 
represented. For example, 1 9.5% of LEAs were estimated to serve 101 to 500 
special education students on the basis of the first-stage sample, compared 
with an estimate of 21 .5% of LEAs based on the second-stage sample. The 
largest discrepancy is in the estimates of the percentage of LEAs serving 1 1 to 
50 special education students; there the relative bias was only 10.5%. 

• The extent to which special education students participated in vocational 
rehabilitation was accurately represented. The largest discrepancy was in the 
percentage of LEAs for which few special education students participated in 
VR; there the relative bias was only 9.2%. 

• The resources provided by both the LEAs and the community were accurately 
represented, with less than a 2% relative bias in either population average. 

LEAs with "up and across" configurations were slightly underrepresented (i.e., 
they were estimated to be 34.6% of the universe using the first-stage sample, 
but were estimated to be only 29.5% of the universe using the second-stage 
sample). LEAs with "out" configurations were slightly overrepresented (6.7% 
using the first-stage sample versus 8.2% using the second-stage sample). 
Neither result was significant at the 95% confidence level, but they were almost 
significant at the 90% confidence level. 

Metropolitan status was accurately represented. The relative errors for 
suburban and rural status were less than 10%. Although the relative error for 
urban status was moderately large (19.9%), the absolute error was extremely 
small (0.5%). 

• The second-stage sample was slightly biased toward LEAs with smaller 
minority enrollment (average minority enrollment was 9.7% using the first-stage 
sample versus 7.9% in the second-stage sample). This result was not 
significant at the 95% confidence level, but was almost significant at the 90% 
confidence level. (Note that in computing "average minority enrollment" large 
and small LEAs were counted equally, rather than being weighted by 
enrollment.) 

LEAs r^erving only grades kindergarten through 8 were underrepresented 
(13.9% using the first-stage sample versus 9.5% using the second-stage 
sample), and LEAs sealing grades kindergarten through 12 were slightly 
overrepresented (82.6% using the first-stage sample versus 86.1% using the 
second-stage sample). These results were significant at 94.9% and 84.4% 
confidence levels, respectively. Several schools serving only grades 
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kindergarten through 8 refused to participate in the study because they 
considered themselves to be elementary districts and had few youth that fell 
into the age group included in the youth sample. As a result, the NLTS sample 
probably underestimates the number of students in grades 7 and 8 by 
approximately 4% or 5%. 

• The percentage of LEAs offering services only to students age 18 or younger 
was slightly underrepresented (11.5% using the first-stage sample versus 8.0% 
using the second-stage sample). This result was not significant at the 95% 
confidence level but was significant at the 90% confidence level. 

These results are encouraging, and it appears that the LEA participants in the study 
sample were not appreciably different from the LEAs from which they were sampled. As 
noted above, only one comparison was significantly different with approximately 95% 
confidence, for reasons that were unlikely to affect the study seriously (there being few 
secondary special education students in the nonresponding K-8 LEAs). A few other 
comparisons were significant at the 90% level, but not more than would be expected by 
chance (i.e., we expected four to five significant results at the 90% confidence level by 
chance alone). 
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nUDENT SAMPLE SELECTION AND ASSOCIATED 
DATA COLLECTION EFFORTS 



The rosters of secondary special education students obtained from participating 
LEAs were used to select a sample of students for which a variety of data collection 
efforts were undertaken. This section describes the procedures used to select the 
student sample, locate parents or guardians, obtain parent/guardian telephone 
interviews, abstract data from school records, obtain data on school and community 
programs, and conduct an in-person survey of nonrespondents to the parent/guardian 
telephone survey. 

Selecting the Special Education Student Sample 

As intended by Congress, the NLTS examines youth with disabilities aged 13 and 
older who were special education students in 1985-86. The study is to provide 
reasonably accurate estimates of what happens to these youth, both overall and for 
each of the 1 1 federal disability categories. To achieve a sample capable of addressing 
overall and within-category analyses, a goal was set of a sample of 1,000 youth for each 
category (except 500 for the deaf/blind), totaling 10,500 for the entire sample. Assuming 
that perhaps as many as 25% would decline to participate in the study at the outset, an 
initial sample of about 14,000 youth was sought, or about 1,320 per condition and 675 
for the deaf/blind. Moreover, using the limited data on attrition available from previous 
studies, this initial size was assumed to be sufficient to result in a final sample of 500 to 
750 after 5 years.* This relatively large sample within disability category permits the 
examination of differences between subgroups (for example, of employment differences 
between males and females) that a smaller sample would be inadequate to address. 

Using the rosters obtained from sampled LEAs, the student sample was stratified by 
disability category and age. For the first variable, the 1 1 federal definitions were used. 
Students were included in the category to which they were assigned by the district or 
state-operated school from which they were sampled in the 1985-86 school year. For 
LEAs that used categories other than the 1 1 federal categories, information on the 
disability of youth was used to reassign them to a federal category. This reassignment 
often entailed calling the state or the LEA in question to confirm the nature of the 
handicap to which their unique label was assigned. For example, the developmentally 
handicapped in Ohio were equivalent to the mentally retarded in the federal 
categorization and, accordingly, were considered to be mentally retarded for sampling 
purposes. 



A sample of 500 completely randomly selected respondents is sufficient to estimate proportions to 
within a standard error of 2.2%. However, in a multi-stage sample such as was used in the NLTS, the 
effective sample size is typically 50% of the actual number of respondents, so a sample of 500 
respondents would allow estimation of proportions to within a standard error of approximately 3.2%. 
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To ensure that sufficient numbers of youth in the sample would have an opportunity 
to move out of school and into adult life, three age categories were defined: 13 to 15, 16 
to 18, and 19 and over (i.e., those born in 1970 through 1972, 1967 through 1969, and 
1966 or earlier, respectively). Most of the first group would be in school for the majority 
of the study period, whereas the latter two would be mainly out of school. The 19-and- 
over group that was still in school was quite small and consisted of youth who were 
primarily moderately to severely handicapped. The 16-to-18 group was more varied on 
severity of handicap. The two student stratification variables (disability condition and 
age) resulted in 33 student strata within each respondent LEA. 

Based on the estimates in OSEP's 8th Annual Report to Congress of national 
distributions of youth by age and by disability category, sampling rates were calculated 
to yield the desired sample size in each disability category and about equal numbers of 
youth in the 13-to-15 and 16-to-18 age categories (approximately 40% of the sample in 
each of these categories) and approximately half this number in the 19-and-older 
category (i.e., 20% of the sample). These sampling rates were multiplied by the counts 
within each of the 33 student strata from each LEA roster to obtain the desired number 
of youth in each student stratum from each LEA. This number of students was then 
randomly selected from the relevant age group within each disability category from each 
roster. For example, if the sampling rate for students in large LEAs who were age 13 to 
15 and classified as mentally retarded was 17%, and an LEA from the large stratum 
reported 36 youth in this age and disability category, then 6 13- to 15-year-old mentally 
retarded youth (i.e., 17% x 36) were sampled from that LEA. Midway through the 
sampling process, it became apparent that previously developed sampling fractions 
were insufficient to generate enough youth in some categories (e.g., speech impaired 
and other health impaired). Sampling fractions were doubled in these categories in the 
remaining LEAs to generate a larger pool of youth. 

Locating Parents/Guardians 

After the students were selected, an effort was launched to locate their parents or 
guardians. This was important for two reasons. First, the initial interview was to be with 
parents/guardians, rather than students. Second, as part of its overall effort to 
encourage districts to release educational information from student school records, SRI 
wanted to obtain written permission to access these records from as many parents/ 
guardians of selected students as possible. 

For parents/guardians in all but 32 school districts where confidentiality concerns 
prohibited staff from giving us a list of names of special education students from which to 
sample, SRI tried to locate parents/guardians directly using the addresses and 
telephone numbers provided by the districts. In these cases, the initial parent contact 
involved a letter explaining the study and informing parents that they would be called for 
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an interview. A consent form was included with the letter giving permission for SRI to 
gain access to school records. A postcard was included for parents to use in informing 
SRI of their current address and telephone number. Repeated follow-up mailings and 
telephone calls were made to nonrespondents. 

In the 32 school districts where confidentiality concerns prohibited district staff from 
giving us a list of names of special education students from which to sample, the district 
sent participation requests to the families. Parents/guardians from these districts who 
did not return the consent form were lost from the sample. 

Table 4 presents the results of our efforts to locate parents/guardians. Of the 12,833 
students selected for the sample, 1 ,632 were nonparticipants in the study because their 
names were never revealed by the districts from which they were sampled and parents did 
not return a consent form. The address information for 636 youth was found to be 
incorrect, with no fonvarding address available from the post office and no working 
telephone number available to call the family to request new address information. Parents 
reported the deaths of 43 students selected for the sample. Thus, parents/guardians of 
1 0,369 youth were available for interview. 

Of the 10,369 parents/guardians available for interview, 4,493 provided written 
permission to access school records, and an additional 1,301 provided verbal consent. 
A small percentage refused consent either verbally or in writing; the remainder did not 
respond. However, SRI tried to obtain telephone interviews from all parents/guardians 
whether or not consent forms were obtained, except for students in LEAs that prohibited 
us from contacting any parents who had not returned a consent form for access to 
school records. 

Table 4 

RESULTS OF EFFORTS TO LOCATE PARENTS/GUARDIANS 



Response to Efforts 


Number 


Desired sample 


12,833 


No location information provided by school district and 




consent form not returned by parent/guardian 


1,632 


Inaccurate location information; unable to locate by mail 




or telephone 


636 


Youth deceased 


43 


Available for interview 


10,369 a 



^ Excluding the 43 deceasGd youth, the rate for efforts to locate parent/guardians Is 1 0,369 / 1 2,780 « 
81 .1%. Table 6 provides a breakdown of the numbers in Table 4 by disability group. 



Fortunately, an unwillingness to return a signed consent form did not imply an 
unwillingness to participate in the study. A large number of parents who did not return a 
form completed the interview. Also, more than half of those who had initially refused to 
participate altogether also granted the interview. Consent forms were also not required 
for access to records in most school districts, although our ability to attest to requesting 
written consent eased entry in many LEAs and schools. 



Obtaining Parent/Guardian Telephone Interviews 

A major component of the NLTS involved interviewing by telephone the parents or 
guardians of youth in the sample. Because the questionnaire was too complex to be 
administered from a hard-copy format, computer-assisted telephone interviewing (CATI) 
was employed by SRI's telephone interview subcontractor, Chilton Research Services 
(CRS). The majority of these interviews were conducted between June 22 and 
September 15, 1987, although a reduced level of interviewing continued through 
November 8, 1987. These interviews averaged 26 minutes in length. 

Before the telephone interviewing began, we sent letters to parents/guardians of 
sample members to inform them that soon they would be contacted for interviews. A 
postcard was included with each contact letter for parents to use in informing us of any 
change to their current address and telephone number. 

About 1 ,600 postcards were returned. For about 700 youth, letters were returned by 
th? post office with a fonA/arding address indicated, or a telephone call to the parent 
resulted in obtaining a new address. About 300 youth were lost to the sample because 
the contact letter was returned with no new address and no telephone number could be 
found to contact the parent. New addresses were obtained for about 800 youth and new 
telephone numbers for about 650 youth. 

Because a major priority of the study was to maximize the number of youth about 
whom interviews were completed, an unlimited call rule was implemented. Interviewers 
made repeated attempts to locate each parent/guardian through the entire period of data 
collection, rotating the contact attempts through various days of the week and times of 
day. 

Many youth were lost to the sample because they were no longer at the address or 
telephone number we had or because we had never had a telephone number for them. 
The following efforts were undertaken to complete interviews for these youth: 

• Use of directory assistance. For many youth, no telephone number was 
provided by the LEA or school from which they were selected. For others, 
initial mailings or telephone contact attempts revealed that the numbers we had 
were no longer in service or did not reach the desired respondents. In such 
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cases, interviewers used directory assistance in an effort to locate a working 
telephone number. This process was not as successful as hoped because 
LEAs often also did not provide the name of the youth's parents. Because 
parent and youth names often differ, asking directory assistance for a 
telephone number associated with the youth's last name at the last known 
address often did not result in finding the needed number. 

• Contacts with LEAs and schools. In September 1 987. toward the end of the 
interview period, the names of all youth for whom addresses or telephone 
numbers were missing or inaccurate were sent to the LEAs or schools from 
which they had been selected. LEAs and schools were asked to provide 
current information if it was available. If youth were no longer in the district or 
school, they were asked to provide any available information on the youth that 
could assist in finding them (e.g., the name of the district to which the youth 
had transferred). More than 200 districts received these requests; about two- 
thirds responded, providing telephone numbers for 27% of the youth who 
previously had been lost to the sample. 

• Toll-free tek phone number. For approximately 1 ,1 00 youth , no telephone 
number had been provided by the LEA from which they were selected and no 
postcard had been returned by the parent indicating a telephone number. 
Because this was such a large proportion of the total sample, several extra 
efforts were undertaken to enable telephone interviews to be completed with 
them. A postcard was sent to all parents for whom we had no telephone 
number that provided them with a toll-free 800 number and encouraged them to 
call for an interview. A second postcard was sent after about 3 weeks 
reiterating that their participation was important and encouraging them to call. 
Despite these efforts, only 43 interviews were completed by parents initiating 
the interview via the toll-free number. 

• Mall questionnaire. In a further effort to reach the parents of youth for whom 
we believed we had a correct address but no telephone number or an 
inaccurate telephone number, we developed a one-page mail questionnaire 
that contained items related to key outcome variables (e.g., whether the youth 
had completed secondary school, had a job). At the bottom of the 
questionnaire was a space for the parents to indicate their current address and 
telephone number. This was mailed late in the interview period to 2,150 
parents of youth for whom no telephone interview had been completed. 
Completed questionnaires were returned by 323 parents, with about 75% 
containing a new telephone number. Telephone interviewers attempted to 
contact the parents, thanked them for the mailed questionnaire, and completed 
the telephone interview. This process resulted in an additional 121 completed 
telephone interviews. For others, the data from the mailed questionnaires were 
entered into the database as incomplete interviews. 

Table 5 summarizes results of the parent/guardian interviews. Of the 10,369 youth 
for whom interviews with parents/guardians were attempted, interviews were completed 
for 62%,* including 280 interviews completed in Spanish. Partial telephone interviews 
were achieved for 2%, and for another 2%, partial information was obtained through the 



Excluding deceased youth, the overall response rate for locating the parent/guardian and completing 
the telephone interview is 6,438 / 12,790 « 50.3%. 
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one-page mail survey. Refusals were only 3% of the sample. For 2%, no respondent 
was available, and in 1% of cases, a language barrier other than Spanish resulted in a 
failure to complete the interview. For 19% of the sample, no correct telephone number 
was located for the sample member, and in the remaining cases, repeated attempts to 
reach the respondent throughout the interview penod never reached a respondent at the 
telephone number we had. There was little variation in response rates among the 
disability categories of the youth. The largest variation occurred for the percentage with 
complete telephone interviews, which ranged from a low of 56% for youth in the mentally 
retarded category to a high of 66% for youth in the orthopedically impaired, visually 
impaired, and multiply handicapped categories. The second-largest variation occurred 
for the percentage of youth without telephone numbers whose parents would not call toll- 
free, which ranged from a low of 14% for the deaf and multiply handicapped to a high of 
25% for the mentally retarded. 

Abstracting Data from School Records 

One important focus of the NLTS is to describe the school programs of secondary 
students with disabilities. Information was needed regarding courses youth took; the 
settings in which they were served; performance indicators, such as grades and 
minimum competency test outcomes; absenteeism; and supplementary services the 
youth may have received from the school, such as speech therapy or personal 
counseling. Because parents are often not fully aware of these aspects of students' 
programs in sufficient detail for study needs, a process of gathering information from 
students' school records was incorporated into the NLTS. Field test experience 
suggested that recruiting local school staff to provide the information from the school 
records would result in more accurate information at lower cost than would be obtained if 
SRI field staff were used in the hundreds of schools attended by sample youth in the 
1985-86 or 1986-87 school year. 

The student sample contains youth whose most recent year in secondary school was 
either the 1986-87 or the 1985-86 school year. The procedures for identifying the 
schools most recently attended differed, depending on the school year in which the 
youth were most recently enrolled in secondary school. 

Identifying Schools Youth Attended In 1986-87— Aiier determining in the 
parent/guardian telephone interview that the youth had been enrolled in secondary 
school in the 1986-87 school year, the parent was asked to identify the name and 
location of the most recent secondary school the youth attended. In many cases, the 
parent did not provide enough information to allow the school to be unambiguously 
identified. Several steps were taken to clarify incomplete or inaccurate school names 
and addresses. These steps included the use of Market Data RetrievaPs school guide 
and the Curriculum Information Center's State Guides for School Year 1986-1 987, use of 
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directory assistance, and contacting LEAs from which students were sampled. In fewer 
than 2% of the cases, the school name given by the parent in the interview could not be 
linked to any identifiable school. 



Identifying Schools Youth Attended In yfl55-86— Parents who said youth were not 
enrolled in secondary school in the 1986-87 school year were not asked what schools 
the youth had attended previously, because the original study design did not call for 
collecting student record or school survey information for those students. However, in 
the course of interviewing parents, a design change resulted in the need to collect this 
information for all youth for their most recent school year. To identify the secondary 
school attended in 1985-86 by youth who were not in school in 1986-87, we reviewed 
the roster of students from which the youth had originally been sampled; in many cases, 
these listed the school in which the youth was enrolled that year. For youth for whom 
the 1985-86 school was not listed on the roster, we asked the record abstractor recruited 
for that district to search district records to identify the school the youth had attended. 

At the conclusion of these efforts, special education students selected from the 303 
participating districts and 22 participating state-operated schools were found to be 
attending a total of 1,810 schools, 400 of which were outside the original sample of 
districts and state-operated schools. 

In the fall of 1987, our contacts in each district/school from which youth were 
sampled were asked to review a list of schools parents reported as attended by sample 
youth and to verify the school address information. A list of sampled youth enrolled in 
secondary school for whom no school had been identified was also sent with a request 
for the contact person to supply the name and location information of the school 
attended by each student. Contacts were also asked to identify an abstractor (or several 
abstractors) who would be willing and qualified to collect student record information for 
students in the sample (abstractors were paid $6.50 per completed abstract). We also 
included an updated version of a letter from the U.S. Department of Education, office of 
the Family Educational Rights and Privacy Act (FERPA), stating that for the purposes of 
this study, educational agencies could release student information to SRI without 
violating federal law. 

Follow-up procedures involving reminder letters and calls were made to about 90% 
of the districts/state-operated schools. The telephone calls were very important in 
assuaging district and school staff concerns about participation. Their two primary 
concerns were: 

• Confidentiality of student records. In the majority of cases, the FERPA letter 
was sufficient for collecting student record information. The FERPA letter did 
not satisfy the concerns of 19 districts and 3 state-operated schools; in these 
cases, we sent copies of the signed parent consent forms that were available. 
Because we did not have parent consent forms for all students, the consent 
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form requirement reduced the response rate for the record abstracts from these 
districts/schools. 



• Burden on district/school staff. We discussed with district personnel how 
student record abstract data could feasibly be collected in that district. District 
staff were encouraged to use as many abstractors as they thought necessary 
to reduce the burden on any one staff member. 

As mentioned earlier, approximately 400 schools outside the original sample were 
identified as serving students in the sample in the 1986-87 school year. Recruitment of 
these schools paralleled recruitment of schools in the original sample, except that the 
principals of these schools were contacted directly to recommend an abstractor. 

Abstractors were recruited in 95% of the original participating districts, 100% of the 
original participating state-operated schools, and 80% of the schools not in the original 
sample of districts. In the majority of districts and schools in which an abstractor was not 
identified, staff did not explicitly refuse to participate; however, they failed to identify 
someone who would serve as an abstractor after repeated requests. 

After repeated reminder mailings and telephone calls, Student Record Abstract 
Forms were returned for 6,241 students, a 60.2% response rate of the 10,369 students 
whose parents/guardians could be located and a 48.8% response rate of the 12,790 
originally selected parents/guardians (excluding deceased youth). 



The School Program Survey 

The Survey of Secondary Special Education Programs was designed to collect 
information about each youth's secondary school, as well as information about the 
community in which the school was located. Section A of the survey contained 
background items concerning the school's students and staffing, and instructions stated 
that this section was best completed by someone with a schoolwide perspective. The 
remainder focused on the school s special education programs, and instructions stated 
that this section was best completed by someone with a good day-to-day knowledge of 
these programs (usually a special education staff member). The survey instrument was 
mailed to the school principal. 

The process for contacting both original districts/schools from which youth were 
sampled and new schools to which youth had moved since the sample was selected 
was similar to that used to establish contact for abstracting information from school 
records. Follow-up procedures included postcards, letters, and telephone calls. In 
addition, a telephone version of Section A of the school survey was developed, and 
schools that had not returned a completed mail survey were contacted by telephone. 
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Of the 1 ,81 0 correctly mailed school questionnaires, 1 ,432 (79%) were completed 
and returned. (An additional 101 questionnaires had been incorrectly mailed, either to 
schools where there were no sample students attending in 1986-87 or to schools that 
were not secondary schools.) Of the 378 schools that either refused or never returned a 
completed school questionnaire, partial school background information was collected by 
telephone for 304 schools. In total, these data provided complete school program 
survey information for 5.81 1 youth who attended school in 1986-87 and partial 
information for 861 youth who attended school in 1986-87. In addition, our sample 
included 957 youth who had last attended in 1985-86 one of the schools that had 
completed the school program survey for 1986-87. Under the assumption that school 
programs had not changed substantially from 1985-86 to 1986-87, we imputed the 1986- 
87 school program survey data to these 957 youth. 

The Nonresponse Survey 

Of the 12,833 students selected for the sample, 6,658 parent/guardian telephone 
interviews were completed or partially completed. This relatively low response rate 
(51.9%) raised the issue of whether those for whom we had completed or partially 
completed telephone interviews were representative of the special education population 
to which the study was intended to generalize. To determine whether bias existed, SRI 
subcontracted with the Research Triangle Institute (RTI) to locate and conduct in-person 
interviews with a subsample of parents/guardians who did not complete the telephone 
interviews. By comparing data from the in-person interviews with data from telephone 
interviews, we could identify whether the telephone interview nonrespondents differed 
systematically from telephone interview respondents. As discussed later, in performing 
the weighting, we combined the responses to the in-person parent/guardian survey with 
the responses to the telephone parent/guardian survey, and used knowledge of the 
differences between the surveys to modify the weights. 

Design 

Not all of the 6,175 youth whose parents/guardians did not complete the parent 
interview were eligible for participation in the nonresponse survey. The ineligible youth 
included the following: 

• Youth whose names were not revealed by the school district and whose 
parents/guardians never returned consent forms. 

• Youth for whom address information was 'ound to be incorrect, with no 
forwarding information available from the post office and no working telephone 
number available to call the family to request new address information. 
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• Youth who were deceased. 

• Youth whose parents/guardians actively refused to participate in the telephone 
survey. 

• Youth for whom there was a language barrier other than Spanish, 

• Youth for whom there was no adult who could respond about the youth. 

• Youth in institutions. 

• Youth in the deaf/blind disability condition. 

In addition, the nonresponse survey could not address the representativeness of 
youth from school districts that did not participate in the NLTS. 

Excluding the ineligible youth, there remained approximately 3,000 youth who were 
eligible for the nonresponse survey. These youth could be classified into one of four 
groups: 

• Youth for whom the address appeared to be correct, but the.telephone number 
was incorrect; i.e., it did not work or did not reach the desired parent/guardian. 

• Youth for whom the address appeared to be correct, but there was no 
telephone number. 

• Youth for whom the address and telephone number both were incorrect. 

• Youth for whom the address and telephone number both appeared to be 
correct, but no one answered the telephone in repeated attempts over the 
several months of the telephone interview period. 

These four groups constituted approximately 31%, 27%, 21%, and 20% of eligible 
respondents, respectively. 

The nonresponse survey sample design was a two-stage cluster sample, m the first 
stage, LEAs (but not state-operated schools) were selected with probability proportional 
to the number of nonrespondents in \he LEA. This sampling procedure tended to yield 
larger districts, which maximized the efficiency of in-person contact. The first-stage 
sample yielded 27 LEAs with a total of 1 ,608 nonrespondents. The second-stage 
sample of students was selected from these 27 LEAs with probability proportional to 
first ctage weights, so that nonresponding students had approximately equal 
probabilities of selection overall. The second-stage sample resulted in a total sample of 
554 youth in the nonresponse survey. 
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Instrumentation 



The parent/guardian telephone survey questionnaire was adapted for use in an in- 
person interview. The contents of the telephone and in-person questionnaires were very 
similar; changes largely were related to questionnaire format. Interviews lasted 
approximately 25 minutes. 



Procedures for Locating Nonrespondents 

Interviewers were supplied with key information on each sample member to assist in 
completing an interview. Key information included the youth's name, the 
parent/guardian's name, last known address and telephone number(s) of the 
parent/guardian, last known school attended by the youth, the disability category of the 
youth, the youth's age, reasons for nonresponse (e.g., no telephone number, bad 
address and telephone, nonworking number, etc.), and whether a written consent form 
or mail questionnaire had already been returned. 

As a first step in locating respondents, interviewers visited the last known address of 
the youth. Three call-backs were attempted at the home address, if the interviewer 
believed the address was correct but no one was at home, if the interviewer believed 
the address was incorrect, neighbors were contacted to attempt to obtain information 
regarding the new location of the sample youth. 

The school last attended by the sample member was also used as a source of 
information in locating a parent/guardian. Interviewers visited the school, explained the 
nature of th^^ study, indicated that they had spoken with the district-level contact person, 
and requested help in finding an address or telephone number for the student. 

New location information was sometimes known by school personnel, often by 
pulling the student's file. School personnel were also sometimes able to provide other 
types of information that could be useful in locating parents/guardians. For example, 
school personnel were asked whether they knew of any other agency that provided 
service to the student (e.g., the Department of Mental Health, a job training center, etc.); 
the name of a new town or district to which the sample member had moved (a request 
for a transcript from a new district was often a source of this information); the name of 
another school in the district in which the student had enrolled after leaving this school; 
the name of any siblings in the school (the files of siblings sometimes contained a 
correct address or telephone number); if the student was not still enrolled, whether the 
student had graduated, dropped out, or aged out; the name of the parent's or the 
student's employer while the student was enrolled in school; the names of friends of the 
student who might know where he/she was; and the names of teachers in the school 
who might know the whereabouts of the sample member. The interviewer initiated 
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telephone calls to all agencies, schools, employers, friends, and teachers whose names 
were provided by the school in an attempt to locate the parent/guardian of the sample 
member. 

If the school could not provide helpful information, other sources of information were 
used, including the local post office, small businesses in the neighborhood where the 
family might be known, city directories, and Polk directories (which list the residents of a 
city by address and were used to obtain the names and telephone numbers of neighbors 
near the subject's last known residence). 

Response Rate 

Of the 554 nonrespondents included in the nonresponse substudy sample, 
interviews were completed with 441 , for a completion rate of 80%. The process 
revealed that the majority of sample members actually lived at the addresses contained 
in the SRI file but had either no telephones or unlisted numbers, or SRI had not been 
given their number by the original school district. 

Contents of the Combined Database 

Data from the several components of the NLTS were combined into a single 
database for analyses, using the unique student identification number to connect data 
for each student from the parent interview (telephone and in-person), record abstract, 
and school survey with the original sample file. Although the response rate for each 
data collection component was reasonably high, complete data from all three 
components were obtained for a smaller percentage of sample members. Table 6 
indicates the percentage of youth in each federal disability category for whom various 
combinations of data were obtained. 

Data from at least one source were obtained for 8,678 youth, 84% of the sample of 
living youth for whom we ever had location information. However, complete data were 
obtained from the parent interview, school records, and the school survey for only 4,064 
(39% of youth available for contact). Data availability is fairly uniform across the 
disability categories, ranging from 34% of youth in the emotionally disturbed category to 
42% of youth in the visually impaired category having complete data. As discussed 
later, weights were computed for 8,404 respondents who had at least a completed (or 
almost completed) parent/guardian interview or a school record abstract. These 8,404 
respondents were used in wave 1 analyses. 
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NONRESPONSE BIAS IN DATA COLLECTION EFFORTS 



In this section, we examine nonresponse bias in the telephone survey, the school 
record abstracts, and the school program survey. We also examine item nonresponse. 



Comparison of the ln*Person and Telephone Interviews 

The in-person and telephone interviews were compared to better understand the differences 
between telephone interview respondents and those individuals who could not be located 
for the telephone interview but could be Icx^ated for the in-person interview and consented 
to be interviewed. There were three complicating factors in making this comparison. The 
first complicating factor was the bias of the in-person interview toward very large school 
districts. For example, approximately 58% of the in-person respondents came from huge 
districts and only 14% from medium and smaller districts, whereas approximately 30% of 
the telephone survey respondents came from huge districts and 24% from medium or 
smaller districts. The second complicating factor was the different allocaticns of the in- 
person and telephone respondents among disability categories. These different 
proportions resulted partly from random chance and partly from not having any youth 
attending state-operated schools in the deaf, deaf/blind, and visually impaired disability 
categories in the nonresponse survey. For example, youth in the deaf, visually impaired, 
and deaf/blind disability categories accounted for only 13.4% of respondents to the in- 
person interview but 23.6% of respondents to the telephone survey. The third 
complicating factor was that interviewing methods were different, so that differences 
between in-person and telephone results may not be attributable exclusively to 
nonresponse bias. As described below, we were able to adjust for the first and second 
complicating factors, but not for the third. 

To make the respondents to the in-person interview and the telephone interview 
comparable with respect to LEA size and disability category, the data were weighted.* 
Our first choice was to weight each data set to represent the appropriate universe of 
special education students. However, the 442 in-person respondents were sparsely and 
disproportionately distributed among the possible 3,168 student strata.** 

Even when we performed extensive strata collapsing (as described below), which 
would have allowed such weights to be calculated, the effect would have been to reduce 



The weighting procedures used to compare the in-person and telephone interviews should not be 
confused with the procedures used to weight the combined database, consisting of in-person and 
telephone parent interviews, school record abstract respondents, and school program survey 
respondents. The procedures used to weight the combined database are discussed later in the report. 
** There were 96 LEA strata (defined by the intersection of 4 geographic regions, 6 sizes, and 4 wealth 
ranges) multiplied by 33 youth strata (defined by the intersection of 3 age groups and 1 1 handicapping 
conditions) for a total of 3,168 strata. 
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the effective sample size* for the in-person survey to only about 100 respondents. 
Hence, we weighted the telephone survey respondents to be comparable to the in- 
person survey respondents via the following weighting process: 

On the basis of an analysis of the telephone survey (described later), the 
3,168 original strata were collapsed. The small and very small district size 
strata, the two lowest wealth strata, the two highest wealth strata, and the four 
geographic regions were collapsed. This resulted in 330 strata defined by 
LEA size (5 levels), LEA wealth (2 levels), disability category (1 1 levels), and 
youth's age (3 levels). 

The respondents to the in-person and telephone interviews were classified into 
the 330 strata. Youth sampled from state-operated schools for the deaf and 
blind were excluded from these strata. 

The telephone interview respondents were weighted to the number of in- 
person interview respondents in each of the 330 strata. For example, there 
were 62 telephone interview respondents and 12 in-person interview 
respondents among youth classified as learning disabled, born in 1970 
through 1972, and attending huge and wealthy LEAs. Each of the 62 
telephone interview respondents was given a weight of 12/62 = 0.194 and 
each of the 12 in-person interview respondents was given a weight of 1 .0. 

The first two columns of numbers in Table 7 compare the characteristics of 
respondents to the in-person interview with those of respondents to the telephone 
interview using the weighting procedure described above. (The last column is described 
later in this section and allows a comparison using another weighting procedure.) The 
youth for whom parents/guardians responded to the in-person interview were more likely 
to be black, Hispanic, American Indian, or Alaskan native (64% in the in-person interview 
vs. 48% in the telephone interview), more likely to live in a one-parent household (54% 
vs. 40%), more likely to come from a household in which the head of household was a 
high school dropout (57% vs. 39%), less likely to come from a household in which the 
head of household had at least some college education (14% vs. 27%), more likely to 
come from a household with an income of less than $12,000 per year (56% vs. 29%), 
less likely to come from a household with an income of at least $25,000 per year (8% vs. 
40%), less likely to be currently enrolled in school (55% vs. 67%), less likely to have 
graduated (50% vs. 71%), and more likely to ha\'e dropped out (42% vs. 19%). The 
graduation and dropout rates were both computed using a base of those no longer in 
school. For purposes of this comparison, dropouts were students who were on the 
school roster in 1985-86 but by the 1987 interview their parents reported that they were 
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The precision of sample estimates is a function of the sample size and sampling weights. Precision 
increases when the sample size increases and tends to decrease when the variance of the sampling 
weights increases. The effective sample size is the number of completely randomly selected 
respondents (i.e., selected without stratification) required to achieve the same precision as that 
available with stratified (and differentially weighted) respondents. 
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Table 7 

COMPARISON OF IN-PERSON AND TF' IPHONE RESPONSES 
TO THE PARENT/GUARDIAN QUESTIONNAIRE 
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3.70 


Self-care ability scale 
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10.43 


10.37 


Functional mental skills scale 


13.29 


12.84 


12.80 


Household-care ability scale 


10.91 


10.10 


9.78 



^ Telephone respondents weighted to distribution of in-person respondents by LEA size and wealth and youth 
handicap and age. 

^ Telephone respondents weighted by factors listed previously and by in-person respondent household wealth. 
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no longer in school, had not graduated, had not been expelled, and had not exceeded 
the age limit for enrollment. 

We investigated whether further weighting of the telephone survey respondents 
could be used to minimize the differences between the telephone and in-person versions 
of the parent/guardian survey. The largest discrepancy between the two surveys was 
with respect to household income, and we hypothesized that all of the observed 
differences would be related to household income. Consequently, we redefined the 
strata into which respondents were entered by dividing each stratum into three substrata 
according to whether household income was less than $12,000 per year, between 
$12,000 and $24,999, or at least $25,000. This defined 990 strata. Otherwise, 
weighting was performed as described above. The in-person interview was unweighted, 
and the telephone interviews were weighted so that within each stratum they summed to 
the number of in-person interviews in that stratum. The effect on telephone survey 
responses of adding household income as a weighting variable is shown in the last 
column of Table 7. The income adjustment substantially improved the agreement 
between the two surveys for all the demographic variables. The proportion of black, 
Hispanic, American Indian, and Alaskan native youth in the reweighted telephone survey 
increased to 59% (vs. 64% for the in-person survey); the proportion of households in 
which the head of household was a high school dropout decreased to 50% (vs. 39%); 
and the proportion of households in which the head of household had at least some 
college education decreased to 19% (vs. 14%). Of course, after income weighting there 
were no remaining differences between the household income distributions of the in- 
person and telephone surveys. The income adjustment did not fully adjust for 
differences in the school status variables. The proportion of youth currently enrolled 
remained unchanged at 67% (vs. 55% among in-person respondents); the proportion of 
youth who had graduated among those no longer in school decreased only slightly to 
69% (vs. 50%); and the proportion of youth who had dropped out among those no longer 
in school increased only slightly to 21% (vs. 42%).* We concluded that some, but not 
all, of the differences between the two surveys was attributable to difficulty in contacting 
lower-incone families by telephone. Additional differences appeared related to difficulty 
in contacting families of youth who were no longer in school, and particularly those who 
had dropped out. 



These differences in school completion status are substaiUial. However, we did not want to weight the 
sample by school completion status to adjust for these differences because school completion status 
was an important outcome variable. Fortunately, the school record abstract also contained information 
on school completion status. As explained later in this report, when these survey results were added to 
the telephone and in-person survey results, and appropriate income-based weights were applied, 
school completion status could be "correctly" estimated to within 1%. The "correct" estimate was 
derived by using the in-person interview respondents to estimate school completion status for telephone 
interview nonrespondents and using telephone interview respondents to estimate the status of the 
remainder of the population. 
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Nonresponse Bias in the Scliooi Record Abstracts and 
School Program Survey 

The school record abstracts and the school program survey exhibited response rates 
comparable to those of the parent/guardian telephone interviews. Consequently, we 
also examined these components of the project to assess the possible extent of 
nonresponse bias. Unfortunately, because there was no nonresponse survey for the 
school record abstracts and school program survey, only indirect inference concerning 
nonresponse bias for these data collection instruments is possible. The analysis 
consists of comparing parent/guardian responses of youth who did and who did not have 
student record abstracts and school program surveys. 

Table 8 compares school record abstract and school program survey respondents 
and nonrespondents with respect to the responses provided by the parents/guardians of 
these youth. The first column lists the items in the parent/guardian survey used for 
comparison. The second column displays the results for the 7,357 students with a 
complete or almost complete parent/guardian interview. Of these 7,357 students, 2,614 
lack either a school record abstract or a school program survey, or both. Table 8 shows 
that 61 .2% of students were males (on an unweighted basis) and 85.9% lived with a 
parent. The third and fourth columns show results for the 1 ,625 and 5,732 students 
without and with a school program survey, respectively, among the 7,357 students with a 
parent/guardian interview. The fifth and sixth columns show results for the 2,402 and 
4,955 students witho"t and with a school record abstract, respectively, among the 7,357 
students with a parent/guardian interview. The last column displays results for the 4,743 
students with all three survey instruments. 

Comparison of the first and last g: Oups of youth in Table 8 demonstrates the overall 
extent of bias associated with the lack of either a school record abstract or a school 
pi ogram survey, because the first group is the least restrictive (i.e., includes all youth 
with a complete or almost complete parent/guardian interview) and the last group is the 
most restrictive (i.e., includes only youth with all three data collection instruments). Only 
a very modest bi::s is demonstrated. The items exhibiting the largest differerioes are 
Caucasian race (63.4% for the least restrictive group versus 66.2% for the most 
restrictive group), currently enrolled in school (62.5% vs. 66.8%), graduated (72.9% vs. 
76.5%), dropped out (18.3% vs. 15.7%), and belonged to school or community group in 
the last year (40.7% vs. 43.6%). 
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Table 8 

COMPARISON OF PARENT/GUARDIAN RESPONSES FOR STUDENTS 
WITH AND WITHOUT THE STUDENT RECORD ABSTRACT 
AND SCHOOL PROGRAM SURVEY 



With 
Parent/ 
Guardian 



Response to Parent Questionnaire Interview 



Number of youth 7357 

Percentage of youth: 

Male 61 .2 

Lives with parent 85.9 

Lives alone 1 .3 

Lives with spouse or roommate 2.8 

Lives with family member or friend 2.7 

Lives in residential or boarding school 2.5 

Lives in a college dorm 0.5 

Lives in military housing 0.2 

Lives in a supervised group home 2.0 

Lives in a mental health facility 0.5 

Lives in facility or institute for disablec* 1 .2 

Lives in a correctlorial facility 0.3 

Black 23.9 

White 63.4 

Hispanic 10.3 

American Indian or Alaskan Native 0.8 

Asian 1 .6 

Youth now enrolled in school 62.5 

Graduated 72.9 

Dropped out 18.3 

Temporarily suspended 0.6 

Permanently expelled 1 .0 

Older than age limit 7. 1 

Youth gets paid for work outside home, school 29. 1 

Any postsecondary education 21 .0 

Belonged to school or other group in last year 40.7 

Lives in a one-parent household 34.1 

HOH^ education: 1 1th grade or less S^-l 

HOH education: high school diploma 33.9 

HOH education: < 4 years of college 1 7.4 

HOH education: 4-yr degree or beyond 1 4.6 

Head of household has a paid job now 74.3 

Household Income < $12,000 26.3 

Household income ^ $1 2,000 and < $25,000 29.6 

Household income ^$25,000 44.1 

Youth abl€ to answer questions for self 71 .7 
Mean values on: 

Average hourly wage If paid ($) 3.72 

Self -care ability scale 1 1 .0 

Functional mental skills scale 1 3.0 

Household -care ability scale 10.0 



Without 
School 
Program 
Survey 


With 
School 
Program 
Survey 


Without 
School 
Record 
Abstract 


With 
School 
Record 
Abstract 


With 
Three 
Survey 
Instruments 


1 odo 
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61.4 


61.2 


62.1 


61.0 


60.3 


77.8 


87.8 


82.6 


87.3 


87.9 


2.5 


1.1 


1.9 


1.1 


1.1 


5.1 


2.3 


3.6 


2.5 


2.3 


3.8 


2.5 


3.2 


2.5 


2.5 


2.4 


2.5 


2.5 


2.5 


2.4 


0.7 


0.5 


0.4 


0.5 


0.5 


0.4 


0.1 


0.2 


0.1 


0.1 


2.8 


1.9 


2.5 


1.8 


1.7 


1.9 


0.2 


1.2 


0.3 


0.3 


2.4 


0.9 


1.7 


0.9 


0.9 


0.3 


0.3 


0.2 


0.3 


0.3 


30.9 


22.3 


28.1 


22.1 


22.1 


53.7 


65.7 


56.3 


66.4 


66.2 


12.6 


9.8 


13.0 


9.2 


9.4 


0.7 


0.8 


0.7 


0.8 


0.8 


2.2 


1.5 


1.9 


1.5 


1.5 


38.5 


68.2 


53.5 


66.4 


66.8 


66.5 


75.7 


67.7 


75.9 


76.5 


23.4 


16.2 


22.3 


16.1 


15.7 


0.5 


0.7 


0.7 


0.6 


0.7 


0.9 


1.1 


1.0 


1.0 


1.0 


8.7 


6.4 


8.4 


6.3 


6.2 


27.6 


29.4 


27.7 


29.6 


29.7 


22.5 


20.3 


20.3 


21.3 


21.7 


30.5 


43.0 


34.9 


43.2 


43.6 


40.4 


32.7 


37.7 


32.6 


32.7 


40.4 


32.7 


37.6 


32.6 


32.5 


33.7 


33.9 


33.3 


34.1 


34.1 


13.9 


18.2 


15.5 


18.2 


18.3 


11.9 


15.2 


13.6 


15.1 


15.1 


67.7 


75.7 


71.2 


75.6 


75.7 


35.0 


24.4 


30.2 


24.7 


24.7 


29.5 


29.6 


30.1 


29.4 


29.4 


35.5 


45.9 


39.7 


45.9 


45.9 


69.6 


72.2 


72.0 


71.6 


71.4 



3.87 


3.69 


3.82 


3.69 


3.68 


11.0 


11.0 


11.0 


11.0 


11.0 


13.1 


12.9 


13.0 


13.0 


13.0 


10.3 


10.0 


10.1 


10.0 


10.0 



Head o( household. 
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Although the overall effect of bias is relatively small, there are important differences 
between those with and without a school record abstract and school program survey. 
These differences are consistent across both instruments, and are as follows: 
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Current enrollment In school. Youth missing either the school record 
abstract or the school program survey were much less likely to be currently 
enrolled in school. For example, only 38.5% of youth missing a school record 
abstract were currently in school versus 68.2% of youth who were not missing 
a school record abstract. Presumably, it was much easier to locate student 
records and to convince school administrators to fill out the school program 
survey when youth were currently enrolled. 

School completion status. Youth missing either the school record abstract or 
the school program survey and who had left school were more likely to have 
dropped out and less likely to have graduated. For example, only 66.5% of youth 
who were missing the school record abstract and had left school graduated, 
versus 75.7% for youth who were not missing the school record abstract. 

Living arrangements. Youth missing either the school record abstract or the 
school program survey were less likely to live with a parent. Presumably, this 
is partially related to the lower school enrollment rate. 

Household demographics. Youth missing either the school record abstract or 
the school program survey were more likely to be black or Hispanic and less 
likely to be white, more likely to come from one-parent households, less likely 
to reside in a household where the head of household was employed, and 
more likely to reside in a household earning less than $12,000 per year. 
Presumably, these household demographics also are associated with lower 
schoo' enrollment rates. 



Item Nonresponse 

Item response rates on all three data collection instruments (the parent/guardian 
interview, the school record abstract, and the school program survey) were very high. 
For the parent/guardian interview, the item with the lowest response rate (87.6%) was 
household income, presumably because this item was viewed by some respondents as 
being personal and unrelated to their child's disability. Other items on the parent/ 
guardian questionnaire had response rates in excess of 90%, and typically higher than 
95%. On the school record abstract, the items with the lowest response rates were 10 
and the number of days absent from school. Other items had higher response rates, 
typically in excess of 90%. Response rates for items in part A of the school program 
survey were all in excess of 90%. Unweighted response rates for household income, IQ, 
and numbers of days absent are presented in Table 9. We conclude that, with the 
exception of IQ, item response rates are acceptable. However, when rates of item 
nonresponse are combined with the response rate for each instrument, the percentage 
of youth who were originally available for interview and actually had income, 10, and 
absenteeism data ranged from 58% (household income) to 42% (10). 
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WEIGHTING THE COMBINED DATABASE 



This section describes the procedures used to weight the combined database. First, 
we discuss the decisions that affected our choice of a weighting procedure. Second, we 
briefly describe the five steps in weighting the database. Next, we discuss in detail one 
of the steps that involved deriving an income adjustment factor related to the in-person 
parent/guardian interview. 

Decisions Affecting the Weighting Procedure 

Weighting of survey data is often a relatively simple process wherein each sample 
respondent is weighted inversely to the probability of sample selection, with an 
adjustment for nonresponse. Weighting of the NLTS data was more complicated for four 
reasons. First, there were multiple instruments, with overlapping sets of respondents— 
4,955 youth had both a parent/guardian interview and school record abstract; 1,270 
youth had a school record abstract but no parent/guardian interview; and 2,179 youth 
had a parent/guardian interview but no school record abstract. Second, there was a 
nonresponse survey that indicated that respondents differed systematically from 
nonrespondents. Third, the high stratification of the universe (into 3,168 strata) and the 
given response rates resulted in many strata without respondents to one or both 
surveys. Fourth, for some of the low-incidence disabilities (deaf, visually impaired, 
orthopedically impaired, and deaf/blind), the customary weights of respondents in the 
smaller LEA strata would have been very large relative to the weights of respondents for 
larger LEA strata, dramatically reducing the effective sample size of the survey. For 
these reasons, we made a number of minor adjustments to the usual weighting 
procedures. 

A procedure for weighting the combined database was developed that incorporated 
the following decisions: 

• Sufficient personal data. Weighting was restricted to students with sufficient 
personal data for meaningful analysis. This excluded respondents who had 
only the school program survey (which related to the general special education 
program, rather than to the courses and services actually provided to the 
student) and students with very sparse parent/guardian interview and school 
record abstract data. 

• Single set of weights. A single set of weights was developed for all 
respondents with sufficient personal data for meaningful analysis. The 
alternative to development of a single set of weights was to develop separate 
sets of weights for the school record abstracts and parent/guardian survey, a 
third set of weights for variables that were developed using responses to 
questions common on both instruments, and a fourth set for analyses that 
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required data from both instruments. This number of different weights would 
lead to confusion on the part of analysts, reviewers, and users of reports 
generated from the data.* (For example, the marginal distributions of each 
variable on a cross-tabulation involving two variables from different instruments 
would not have been consistent with the marginal distributions of those 
variables computed using single-instrument weights.) 

• Adjustment for nonresponse bias. To the extent possible, the weights for 
the respondents were developed to compensate for known nonresponse 
biases. Sources of information concerning nonrespondents to the telephone 
survey included both the 442 in-person parent/guardian interviews and 1 ,270 
school record abstracts of additional youth for whom a parent/guardian survey 
had not been administered by telephone. 

• Pooling of youth. For all LEAs in a particular LEA stratum, youth within the 
same disability category and age substrata received equal weights. That is, the 
weights for youth were calculated as if within each LEA stratum we had 
compiled a single list of youth with a particular handicap and age group and 
had selected sample members randomly from that list. (This contrasts with our 
actual procedure of selecting sample members from each separate LEA list 
using comparable sampling rates.) 

Collapsing of strata. Strata with similar responses or similar sampling rates 
were collapsed to reduce the variability in weights associated with random 
nonresponse. 

• Reduction of large weights. Weights for youth in the deal, visually impaired, 
and orthopedically impaired categories in the smaller LEA strata were reduced 
marginally to increase the effective sample size. Weights for youth in the next- 
larger LEA strata were correspondingly increased to reduce bias. Three 
deaf/blind respondents in the medium LEA strata were discarded. The 
universe for the deaf/blind was defined to include only those deaf/blind youth 
who attended one of the state-operated schools or LEAs with an enrollment of 
at least 10,000. 



Weighting Procedure 

The weighting procedure consisted of five steps. 

Step t— The first step was to construct a file consisting of all youth with sufficient 
information for meaningful analysis. This file consisted of youth with at least one of the 
following: (1) a complete or almost complete parent/guardian telephone interview, (2) a 
complete or almost complete parent/ guardian in-person interview, or (3) a school record 



The responses to the parent/guardian telephone interviews were available before responses to other 
data collection instruments. Because we wanted to perform preliminary tabulations using these 
responses, we calculated an early set of weights for telephone interview respondents. The results from 
these tabulations were quite similar to those obtained with the final set of weights (which considered all 
data collection instruments). On this limited basis, we conjecture that the use of multiple weights would 
not have substantially affected estimates. Nevertheless, a proliferation of different weights and 
corresponding estimates that were similar but not identical would have been confusing. 
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abstract. If a youth had no abstract but did have a partially completed parent/guardian 
Interview, that interview was reviewed on an individual basis to determine whether it was 
sufficiently complete to allow inclusion of the youth in the file. Youth having only school 
survey data were not weighted because the school survey provided no individual-level 
data. As mentioned earlier, the final file contained a total of 8,404 youth. 

Step 2— The second step was the collapsing of the sampling strata to facilitate 
weighting by reducing the number of cells with no respondents and to reduce the 
variability in weights attributable to random nonresponse. Our initial analyses had 
shown that among stratifying variables, response variability was mostly a function of the 
youth's disability category and age group. Therefore, these strata were not collapsed. 
On the other hand, the LEA strata had a surprisingly small effect on responses. For 
example, the average intracluster correlation of all youth within an LEA over a set of 
important variables on the parent/guardian questionnaire* was only 0.036, indicating that 
LEA stratification variables were less important than originally thought. We collapsed 
the LEA strata as follows: 

Geographical regions were collapsed. This stratification variable had been 
used to assure adequate geographic diversity in the sample within each district 
size and wealth substratum. However, the sampling rates within LEA size 
strata were very similar across regions, making it a logical candidate for 
collapsing. In addition, of all the LEA stratification variables, it demonstrated 
the weakest effect on responses, apparently because there was substantial 
diversity within each geographic region. 

• Small and very small LEAs were collapsed. These smaller LEAs provided very 
similar services to students with disabilities, and frequently entered into 
cooperative agreements with larger LEAs for the provision of services.** In 
addition, the sampling rates for these LEAs and youth within these LEAs were 
similar. 

• The two lowest wealth strata were collapsed and the two highest wealth strata 
were collapsed. Sampling rates within LEA size strata for the two lowest 
wealth strata and for the two highest wealth strata were very similar. In 
addition, this variable did not show as large an effect on the response variables 
as we had anticipated. This might be a consequence of the fact that the QED 
data on wealth were based on the 1980 census, whereas data were collected 7 
years later. 



These variables included whether the student did any work for which he/she was paid (other than 
his/her work-study job or work around the house), the hourly wage earned by the student if he/she 
worked, a household-care ability score, an intellectual ability score, a self-care ability score, whether the 
student belonged to a school or community group in the past year, whether the student had any 
postsecondary education, whether the youth was in school in the past 12 months, the student's exit 
status from school (graduated, dropped out, suspended, etc.), and the number of services received by 
the youth in the past year (ranging from 0 to 21). 
** James S. Fairweather, "Analysis of a survey of school-district practices regarding secondary-level 

handicapped students and their transition to postsecondary experiences," Center for the Study of Higher 
Education, The Pennsylvania State University, September 15, 1988. 
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These actions had the effect of reducing the number of LEA strata from 96 (defined 
by 4 geography categories, 6 size categories, and 4 wealth categories) to 10 (defined by 
5 size categories and 2 wealth categories). Within each LEA stratum, there were 33 
substrata for students, defined by 1 1 disability categories and 3 age categories. Thus, 
the total number of strata for youth in LEAs was reduced from 3,168 (i.e., 96 x 33) to 330 
(i.e., 10 X 33). In addition, there were 15 youth strata in state-operated schools, defined 
by 5 disability categories (i.e., deaf, hard of hearing, visually impaired, multiply 
handicapped, and deaf/blind) and 3 age categories. 

Step 3— The third step was the computation of weights in the traditional manner 
within each of the 345 strata, under the assumption that all youth within a stratum could 
be pooled and assigned a common weight. That is, each youth in a stratum was 
assigned a weight given by the following formula: 

Wi = (Ni/ni)x(Mi/mi) 

where i is an index denoting one of the 346 strata, Nj is the number of LEAs in the 
universe in the /th stratum, nj is the number of participating LEAs in the /th stratum, Mj is 
the number of youth with the appropriate disability and age category for the /th stratum 
as listed on the combined enrollment lists of the ni participating LEAs, and mj is the 
number of youth with adequate data for meaningful analysis from among the Mj youth. 

After completing the third step, we calculated the approximate sampling efficiency for 
each disability category. The sampling efficiency was estimated using the following 
formula: 

Effk = (M[wk])2 / [ Var[wk] + (M[wk])2] 

where Effk is the sampling efficiency of the Wh disability category, M[wk] is the mean 
value of the weights of all respondents in the Wh disability category, and Var[wk] is the 
variance of those weights. This formula ignores the very weak clustering effects. An 
efficiency of 50% may be interpreted to mean that a simple random sample of youth in 
the /cth disability category half the size of the actual sample would have yielded equal 
precision. (Variance estimates based on pseudo-replication were later compared with 
those based on this simple equivalence to random sampling, and these comparisons 
demonstrated the appropriateness of the above formula.) 

Sampling efficiencies were lower than desired for four low-incidence disability 
categories (deaf/blind, orthopedically impaired, visually impaired, and deaf). Sampling 
efficiencies for these disability categories were 5%, 26%, 34%, and 41%, respectively. 
These sampling efficiencies were low primarily because of the large weights for the few 
youth in the smaller LEAs. We considered three alternatives: 
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• Tolerate the low effective sample sizes. This alternative was considered 
unacceptable. 

• Change the definition of the universe for these disability categories by removing 
smaller LEAs and all respondents from these LEAs. We chose this alternative 
for the deaf/blind disability category because there were only 3 deaf/blind 
respondents in medium or smaller LEAs. These respondents had weights that 
ranged from 1 0 to 400 times the size of the average weight for the remaining 
100 respondents. Removing these 3 respondents raised the sampling 
efficiency to 85%. For the other disability categories, there were enough 
respondents in smaller LEAs that we felt this action would be imprudent. 

• Reduce the weights for youth in the smaller LEAs. We chose this alternative 
for youth in the orthopedically impaired, visually impaired, and deaf categories. 
In addition, to decrease any possible biases so introduced, we increased the 
weights of youth in next-larger LEAs, so that the sum of the weights over all 
youth in each disability category was unchanged. This action increased 
sampling efficiencies for these youth to 60%, 48%, and 56%. respectively. 

Step 4— As discussed above, the fourth step was the modification of weights for 
youth in the deaf/blind, orthopedically impaired, visually impaired, and deaf categories. 
The effects of these weight modifications on the universe are shown in Table 10. For 
example, the effect of weight modification was to reduce the percentage of deaf students 
in small and medium-sized LEAs from 30.5% to 22.7%, and to correspondingly increase 
the percentage of students in large and very large LEAs from 14.6% to 22.6%. Thus, 
the NLTS sample is weighted to somewhat underrepresent deaf students in small and 
medium LEAs and to somewhat overrepresent deaf students in large and very large 



Table 10 

PERCENTAGE OF YOUTH IN STATE-OPERATED SCHOOLS 
AND DIFFERENT SIZE LEAS BEFORE AND AFTER 
MODIFICATION OF WEIGHTS 



Percentage of Youth Who Were: 



Orthopedically Impaired Visually Impaired 



Deaf 



Stratum 



Before After Before After 



Before 



After 



State-operated 



19.7 
22.6 
14.3 
14.8 
18.3 
10.3 



36.4 
18.5 

4.1 
10.5 
23.1 

7.4 



36.4 
18.5 
10.5 
12.1 
15.4 
7.3 



school 
Huge 



Very large 
Large 
Medium 
Small 



0.0 
28.3 
10.3 
21.3 
19.7 
20.5 



0.0 
28.3 
11.9 
33.5 
14.7 
11.5 



19.7 
22.4 
7.4 
12.8 
25.7 
12.1 



51 



63 



LEAs. We also performed a number of tabulations to ascertain whether these weight 
modifications introduced biases, as shown in Table 1 1 . The amount of bias introduced 
was small in all the variables examined.* 

The weighting procedure that we had implemented to this point still did not address 
the differences that we found between the in-person and telephone interview 
respondents. These differences were both statistically significant and substantial. 
Therefore, we felt that it would be desirable to use the data from the nonresponse survey 
in conjunction with the data from the telephone survey to achieve better estimates of the 
universe than could be achieved by use of the telephone survey respondents alone. Our 
first option for doing this was to weight the in~person responses to represent the 
nonrespondents to the telephone survey. Unfortunately, there were too few respondents 
to the in-person survey, c^nd they were too disproportionately distributed among strata to 
allow them to represent all nonrespondents. To have weighted in-person respondents to 
represent all nonrespondents to the telephone survey would have reduced the effective 
sample size of the survey substantially; in addition, 164 of the 330 strata and an 
additional 15 state-operated school strata had no respondents to the in-person interview. 

Our second option for using the data from the in-person interviews to achieve better 
estimates of the universe was to pool the interviews, regardless of whether they were 
obtained in person or via telephone, and weight the pooled sample to represent the 
universe of special education students. (This is the approximate condition of our 
weighting procedure after the fourth step as described above.) Unfortunately, this 
procedure still would have underrepresented nonrespondents — the nonrespondents to 
the telephone survey were projected to be 30% of the universe, whereas in-person 
interview respondents were only 6% of the combined pool of in-person and telephone 
interview respondents. In addition, 197 strata would have consisted entirely of 
telephone interviewees, so that the adjustment would have been ineffective. 

Step 5— Our third option, and the one implemented in the fifth step, was to estimate 
the household income distribution for each disability category using the in-person and 
telephone interviews, and to adjust the weights of all respondents to achieve this 
marginal distribution. This procedure is described in the next section. These 
adjustments were not large. The weights of respondents with household incomes less 
than $12,000 were multiplied by a factor ranging from 1.30 to 1.59 (depending on 
disability category); the weights of respondents with household incomes between 



The fact that average responses with unmodified and modrfiod weights were similar does not imply that 
the modified weights are more desirable than they would otherwise be; neither would large differences 
imply that the modified weights are less desirable. Weighting must be done correctly, whether or not 
the weighted results change. 
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$12,000 and $24,999 were multiplied by a factor between 0.97 and 1 .07; and the 
weights of respondents with household incomes of $25,000 or more were multiplied by a 
factor between 0.67 and 0 J8. The weights of respondents who did not report household 
income (including those with only record abstracts) were not changed. Because the 
weights of youth in different LEA size strata spanned wide ranges, these weight 
adjustments increased the coefficient of variation* of youth weights by only 1 0% (for the 
speech impaired) to 29% (for the emotionally disturbed). Table 12 compares the results 
obtained before and after income adjustment for youth in three disability conditions- 
learning disabled, mentally retarded, and emotionally disturbed— the latter being the 
condition for which the income adjustments had the largest effect overall. 

Procedure for Developing Income Adjustment Factors 

This section describes the procedures used to develop the income adjustment 
factors that were applied to adjust partially for telephone survey nonresponse bias. The 
first step was to estimate the proportion of the population represented by telephone 
survey respondents and its complement, the proportion of the population that would 
have been nonrespondents had the entire population been selected for the survey. To 
do this, we calculated weights (using the second and third steps of the weighting 
procedure as described in the previous discussion) for all youth who were sampled, 
whether or not they had a completed parent/guardian telephone interview. That is, we 
calculated weights as if the parents/guardians of all sampled youth were respondents. 
Then, within disability category, we summed the weights for telephone survey 
respondents and nonrespondents. These two sums then were divided by the total of all 
weights for the disability category to establish the percentage of telephone survey 
respondents and nonrespondents. For example, as shown in the first two lines of 
Table 13, the percentage of telephone survey respondents in the universe of the 
mentally retarded was estimated to be 63.2%, and the percentage of telephone survey 
nonrespondents was estimated to be 36.8%. 

The second step was to calculate the income distribution for telephone survey 
respondents weighted to the universe of estimated telephone survey respondents (i.e., 
the estimated 64.8% of all special education students whose parents/guardians would 
have responded to the telephone survey had they been selected for the sample). For 
example, as shown in Table 13, we estimated that in the universe of telephone survey 
respondents 28,4% of the mentally retarded would come from households with family 
incomes less than $12,000 per year. 



The coGfticient of variation of youth weights is the standard deviation of youth weights divided by the 
mean of youth weights. Increasing the coefficient of variation tends to decrease the effective sample 
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The third step was to calculate the income distribution for in-person interview 
respondents weighted to the universe of estimated telephone survey nonrespondents 
(i.e., the estimated 35.2% of all special education students who would not have 
responded to the telephone survey had they been selected for the sample). This 
distribution is only approximate, because in many of the disability categories, there were 
no in-person respondents in some of the strata (especially those for state-operated 
schools and smaller LEAs). For example, as shown in Table 13 we estimate that in the 
universe of telephone survey nonrespondents, 68.1% of the mentally retarded would 
come from households with family incomes less than $12,000 per year. 

The fourth step was to calculate the ratio of the two income distributions. For 
example, as shown in Table 13. the ratio of the proportion of nonrespondents in the 
mentally retarded category with household incomes below $12,000 to the proportion of 
telephone survey resporidents in that category with household incomes below $12,000 
was 2.40 = 68.1% / 28.4%. We note that these ratios are quite variable across disability 
categories as a consequence of small sample sizes. 

The fifth step was to calculate an approximate overall ratio of income distributions. 
To do this, we weighted the ratios from each disability category by their estimated 
precision. As shown in the last column of Table 13, we estimated that, overall, the 
proportion of nonrespondents with household incomes below $1 2,000 was 2.44 times 
the proportion of respondents with household incomes below $12,000. Similarly, the 
proportion of nonrespondents with household incomes of $25,000 or more was 
approximately 0.16 times the proportion of respondents with household incomes of 
$25,000 or more. 

The sixth step was to multiply the approximate overall ratios by the income 
distribution of respondents, and then to normalize the result to 100%. The objective of 
this step was to use the overall ratios and the income distribution of telephone survey 
respondents to estimate the income distribution of telephone survey nonrespondents. 
This was considered preferable to using the disability-specific ratios because of the 
small sample sizes in individual disability categories. The results of both parts of this 
step are shown in Table 13. For example, when we weight the parent/guardian 
telephone sur/ey for students classified as mentally retarded to all respondents in this 
disability category, we find that 21 .6% are projected to have household incomes less 
than $12,000 per year. Over all handicapping conditions, we estimate that the 
proportion of nonrespondent households with incomes less than $12,000 is 2.44 times 
the proportion of respondent households with incomes less than $12,000. Thus, we 
could estimate that the percentage of nonrespondents in the mentally retarded category 
with household incomes less than $12,000 is 69.2% = 2.44 x 21 .65%. Similarly we 
could estimate that the percentage of nonrespondent mentally retarded students with 
household incomes between $12,000 and $24,999 is 31 .7%, and the percentage with 
household incomes of $25,000 or more is 6.1%. However, these percentages add to 
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107.0%. If we normalize them to add to 1 00%, we obtain our final estimates of 
nonrespondent household income distribution: 647% less than $12,000, 29.6% 
between $12,000 and $24,999, and 5.7% $25,000 or more. 

The seventh step was to estimate the income distribution by disability category for 
the entire universe. The income distribution for telephone survey respondents (as 
calculated in step 3) was multiplied by the proportion of telephone survey respondents in 
the universe (as calculated in step 1), and this was added to the product of the income 
distribution for telephone survey nonrespondents (as calculated in step 6) multiplied by 
the proportion of telephone survey nonrespondents in the universe (as calculated in step 
1). For example, the proportion of youth in the mentally retarded disability category with 
household incomes below $12,000 was estimated to be 41 .7% = (28.4% x 63.2%) + 
(64.7% X 36.8%). 

The eighth step was to weight all survey respondents with sufficient data for analysis 
to the total population, as described in steps 1 to 3 in the text addressing the weighting 
procedure. For example, as shown in Table 13, the proportion of youth in the mentally 
retarded category from households with incomes below $12,000 was estimated via this 
procedure to be 32.0%. This was less than the 41.7% estimated in step 8 because 
fewer than 36.8% of the parent/guardian interviews of this disability category were from 
in-person interviews. Therefore, we sought to adjust the weights to equalize the income 
distributions calculated in steps 7 and 8. 

The ninth step was the calculation of the necessary weight adjustments. These 
adjustments are the ratio of the income distributions from steps 7 and 8. For example, 
as shown In Table 13, all youth in the mentally retarded category from households with 
Incomes less than $12,000 had their weights multiplied by a factor of 1 .30 = 41 .7/32.0. 
Similarly, youth in the mentally retarded category with incomes of $25,000 or more per 
year had their weights multiplied by a factor of 0.75 = 26.0/34.8. 
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CONCLUSIONS 



In this section we examine how well the NLTS sample performs and provide guidance 
on the sample limitations. First, we examine how well the sample is able to estimate 
school status. Second, we compare the NLTS estimates of the number of students in 
each disability condition in the universe with U. S. Department of Education counts of the 
number of students in each condition. Third, we describe the limitations of the sample. 
Finally, we provide an overall assessment of the usefulness of the sample and the extent 
to which it represents the intended universe. 

Adequacy of Weighting Procedure for Estimating Schoo! Status 

As mentioned earlier in the section that compared the results of the in-person and 
telephone surveys, income adjustment only slightly reduced the differences between the 
two surveys* with respect to the school status variables (i.e., the proportion of youth 
currently enrolled, the proportion of youth who had graduated, and the proportion of youth 
who had dropped out). These are important variables for analysis. Fortunately, there is 
an alternative source of data for these variable?;, namely the school record abstract. The 
1 ,270 youth with only school record abstracts and the 442 youth with in-person interviews 
are both representative of nonrespondents to the telephone survey with respect to school 
status variables. Using the results from the telephone interview alone, we estimated that 
65% of all parents/guardians of special education students would have responded to the 
telephone survey had they been asked and the remaining 35% would have been 
nonrespondents. As shown in the last line of Table 14, youth with only school record 
abstracts or with in-person interviews represented 24% of all special education students 
on a weighted basis. Thus, on an overall basis, youth whose parents/guardians were 
nonrespondents to the telephone survey, but for whom we could othenwise gather data 
on school status, only slightly underrepresented telephone survey nonrespondents. 

Table 14 also shows the extent of underrepresentation for each disability category. 
For example, we estimate that if all students classified as emotionally disturbed in the 
universe had been sampled, 43% of their parents would have been nonrespondents to the 
telephone survey. There are two sources of data for the school completion status of these 
nonrespondents: the in-person interview and the school record abstract. The proportion 
of total weight assigned to respondents for students classified as emotionally disturbed 
with either an in-person interview or only a school abstract is 23.3%. Thus, without benefit 
of income weighting of respondents to the parent/guardian telephone survey, the school 
completion status of more than half of nonrespondents was correctly represented in data 
tabulations. 



In various locations in the text we refer to the parent/guardian telephone survey and the parent/guardian in- 
person survey as two different surveys, even though the survey instruments were very sirriilar, because 
there were differences in the populations being surveyed and the method of survey administration. 
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Table 14 

PROPORTION OF UNIVERSE AND TOTAL WEIGHTS ASSOCIATED 
WITH TELEPHONE NONRESPONDENTS 

Estimated Percentage of Percentage of Total 
Nonrespondents to the Weight Assigned to 

Telephone Survey in the Respondents with Either 

Universe (If Everyone an In-Person Interview or 

Disability Category Were Sampled) Only a School Abstract 



All disabilities 


35.2 


23.7 


Learning disabled 


32.6 


21.4 


Emotionally disturbed 


43.0 


23.3 


Speech impaired 


34.8 


24.2 


Mentally retarded 


36.8 


30.5 


Visually impaired 


38.1 


19.6 


Hard of hearing 


36.0 


14.5 


Deaf 


31.0 


14.5 


Orthopedically impaired 


36.7 


22.2 


Other health impaired 


47.8 


16.4 


Multiply handicapped 


41.8 


19.1 


Deaf/blind 


33.0 


14.3 



To examine further the adequacy of our weighting procedure with respect to school 
status, we compared school status estimated using two different sets of weights. The 
first set was the weights calculated as described above (i.e.. the final weighting 
procedure). The second set used weights calculated via the following three steps: 

Estimate the proportion of youth in the universe whose parents/guardians 
would have responded to the telephone survey if they had been selected, and 
the complementary proportion who would not have responded. 

• Calculate weights for youth with telephone interviews so that they represent all 
youth whose parents/guardians would have responded to the telephone 
interview had the entire universe been selected. 

• Calculate weights for youth with in-person interviews or only school record 
abstracts so that they represent all youth whose parents/guardians would not 
have responded to the telephone interview had the entire universe been 
selected. 

We have termed the second method the **separate** weighting method, because 
weights are calculated for different data collection instruments to extrapolate to separate 
and disjoint portions of the universe. 
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Table 15 shows school status calculated using both of these sets of weights. 
Generally there is very good agreement between these two different weighting methods. 
Almost all estimates differ by fewer than 3 percentage points. The only exceptions are 
as follows: 

• The percentage of students in the deaf/blind category who had graduated is 
estimated to be 21 .5% based on our final weights and 17.8% based on the 
separate weighting procedure. Because there are only 100 deaf/blind student 
respondents and no in-person interviews, this discrepancy is within 
expectations. 

• The percentage of students in the other health impaired category who were in 
school is estimated to be 70.7% based on our final weights and 63.5% based 
on the separate weighting procedure. The percentage who had dropped out is 
estimated to be 7.6% based on the final weights and 13.1% based on the 
separate weighting procedure. 

Overall, there appears to be a very slight bias in the final weighting procedure with 
respect to overestimating the number of students in school and underestimating both the 
number who had graduated and the number who had dropped out. In all but one or two 
disability categories, the number in school is marginally larger with the final weighting 
procedure than with the separate weighting procedure, and the number graduated and 
dropped out is marginally smaller with the final weighting procedure than with the 
separate weighting procedure. 



Comparison with OSEP Annual Report to Congress 

As a further check on the reasonableness of our estimates of the number of special 
education students, we compared the weighted numbers of special education students 
served by LEAs as estimated by the NLTS and as reported by OSEP in the 10th Annual 
Report to Congress, This comparison is shown in Table 16. Three ratios are computed 
at the bottom of this table: the ratio of the numbers of 13- to 21 -year-old students from 
these two sources, the ratio of^the numbers of 14- to 21 -year-old students, and the ratio 
of the numbers of 15- to 21 -year-old students. 

As is seen in the last column of Table 16, the NLTS slightly underestimates the total 
number of 13- to-21 -year-old special education students (i.e., NLTS estimates are only 
94% of OSEP counts), estimates more accurately the number of 14- to 21 -year-old 
special education students, and essentially estimates the correct number of 15- to 21- 
year-old special education students. There are a number of reasons for the differences 
in counts of 13- and 14-year-old students. Only students in grade 7 and above were 
eligible for the NLTS, and some 13-year-olds are in 6th grade. The latter students would 
be included in the OSEP counts but not in the NLTS estimates. We also expect 
differences in the number of 13- and 14-year-old students because, as mentioned in the 
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section on LEA bias, the participation rate was lower among K-8 districts than among 
districts that also served grades 9 and above, introducing a slight downward bias in the 
NLTS counts of younger special education students. For these reasons, the most 
appropriate comparison is between the number of 15- to 21-year-o!ds in the NLTS and 
the figures in the 10th Annual Report to Congress. 

As mentioned earlier, the NLTS accurately estimates the total number of 15- to 21- 
year-old special education students. Within disability categories, the accuracy of NLTS 
estimates appears to vary substantially. The NLTS almost correctly estimates the 
number of 15- to 21 -year-old learning disabled students (i.e., the NLTS estimate is 99% 
of the OSEP count). The NLTS slightly underestimates the number of students with 
multiple handicaps (i.e., the NLTS estimate is 96% of the OSEP count), moderately 
underestimates the number of students classified as other health impaired and 
emotionally disturbed (i.e., 88% arid 87% of the OSEP counts, respectively), and 
substantially underestimates those cla3sified as speech impaired or deaf/blind (i.e., 72% 
and 21%, respectively). The NLTS moderately overestimates the number of students 
classified as visually impaired, orthopedically impaired, and mentally retarded (i.e., 
1 1 5%, 1 1 4%, and 1 1 3% of OSEP counts, respectively) and substantially overestimates 
the combined number of students who were deaf or hard of hearing (i.e., 130%). The 
discrepancy in the number of deaf/blind students is attributable to their scarcity outside 
the state-operated schools. As explained earlier, three deaf/blind students in medium- 
and smaller-sized LEAs were excluded from the NLTS because their weights (which 
totaled approximately 200) were substantially larger than the weights of the other 100 
respondents. Inclusion of these students would have yielded much more comparable (if 
still very unstable) estimates of the number of deaf/blind students outside state-operated 
schools, but would dramatically have decreased the precision of estimates for deaf/blind 
students. 

Other tfian for deaf/blind students, the discrepancies between the NLTS estimated 
numbers of 15- to 21 -year-old students and the OSEP counts are not readily 
explainable. Because responding students are weighted up to the total number of 
students in the appropriate disability category on the LEA rosters, these discrepancies 
cannot be associated with nonresponse rates to either the telephone or in-person 
parent/guardian surveys, the student record abstract survey, or the school program 
survey. That is, approximately the same totals would have been estimated by the NLTS 
whether the response rates to the different data collection instruments had been 
substantially lower or higher than they were.* The obvious potential cause for 
differences in counts would be differential participation rates for LEAs within size strata 
as a function of the number of special education students served. That is, LEAs with 
more (or fewer) special education students than other LEAs in their size stratum might 



The totals would be exactly the same, except for the influence of the income adjustments for non- 
response bias. However, we estimate the effect of the income adjustments to be no more than 1-2%, 
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have been more (or less) willing to participate in the NLTS than other LEAs in their size 
stratum. However, we then would have expected a uniform under- or overestimation of 
the number of special education students across disability categories. Instead, we find 
some categories (such as mentally retarded) overestimated and some (such as 
emotionally disturbed) underestimated. Having ruled out this potential cause, we are at 
a loss to explain these discrepancies (other than attributing them to sampling variability). 

Potential Study Limitations Related to Sampling Issues 

The NLTS data are being made available to a broad community. Making data 
available for use in the public domain entails the risk that they will be used to support 
mistaken interpretations because of a failure to account for sample limitations. These 
potential risks are offset by the benefits to be derived from wide dissemination and use 
of the rich NLTS database to address myriad questions in the field about the 
experiences of youth with disabilities in transition. However, to minimize the potential 
that data will be misinterpreted, the user should keep in mind the following 
considerations related to the NLTS sample: 

• Subgroup definitions. Youth were sampled from disability categories based 
on the primary disability designated by the youth's district in the 1985-86 school 
year. Category definitions, assessment procedures, and rules of thumb for 
categorizing students vary between states and often between school districts 
within states. NLTS data should not be interpreted as generalizing to youth 
who truly had a particular disability, but rather to youth who were categorized 
as having that disability by their school district. 

• Potential bias associated with LEA nonpartlclpatlon. Because the 
participation rate among LEAs in the main study was slightly less than 50%, we 
were concerned whether participating LEAs differed from nonparticipating 
LEAs. Comparison of nonparticipating LEAs with 1,600 LEAs from which they 
were sampled revealed no systematic differences across a wide range of 
variables. However, there is no assurance that nonparticipating LEAs did not 
differ systematically from participating LEAs in other (unmeasured) ways. 

• Potential bias associated with parent/guardian nonpartlclpatlon. Of the 

12,833 sampled youth, 6,694 parent interviews were completed (or almost 
completed) over the telephone. Thus, there was concern that the participating 
parents/guardians differed from nonparticipants. An in-person interview of 554 
nonrespondents (with an additional 441 completed interviews) was conducted 
to determine the extent of parent/guardian nonresponse bias. This substudy 
showed that there were differences between the two populations, which mostly 
could be removed by reweighting the income distribution of all respondents. 
However, there is no assurance that the reweighting procedure eliminated all 
differences between respondents and nonrespondents, as a consequence of 
features of tfie nonresponse survey that render it less than a fully 
representative sample of nonrespondents (i.e., its concentration in urban areas, 
the decision not to conduct in-person interviews with those who had actively 
refused to participate in the telephone interview or in districts that would not 
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provide data on parent names and addresses, and a 20% nonparticipation rate 
in the nonresponse survey). In addition, it is possible tiiat there were 
dimensions other than those that we examined on which participants and 
nonparticipants differed (e.g., other variables or variable correlations). 

• Potential bias associated with Inability to obtain school record abstracts 
and school program surveys. Record abstracts were obtained for 6,225 of 
the 12,833 sample members, and school program surveys were obtained for 
6,672 sample members. Those with and without school record abstracts 
appeared to differ primarily with respect to enrollment status (it being easier to 
locate school records and convince school administrators to complete the 
school program survey when the sampled youth were currently enrolled) and 
secondarily with respect to factors that might explain lower enrollment rates 
(i.e., household demographics) or be associated with leaving school (i.e., 
school completion status of dropping out and no longer living at home). In 
addition, it is possible that there were dimensions other than those that we 
examined on which youth with and without these data collection instruments 
differed. 

• Potential bias associated with weighting multiple Instruments. There were 
three important instruments from which we obtained data: the parent/guardian 
survey (both the telephone version and the in-person version), the school 
record abstract, and the school program survey. There were some overlaps of 
variables in these instruments (particularly with respect to key outcome 
variables), and composite variables were formed when information from two or 
more sources was available. Rather than develop and publish a series of 
weights — one for each instrument and combination of instruhients— we elected 
to develop and publish a single set of weights for all respondents with at least a 
complete or almost complete parent/guardian survey or a school record 
abstract. This greatly simplifies the practical aspects of performing tabulations, 
cross-tabulations, and other analyses. However, it yields suboptimal weights 
for each instrument and combination of instruments. To checl^ the 
reasonableness of this decision, we developed preliminary weights based on 
different combinations of instruments and compared variable tabulations and 
cross-tabulations using these different sets of weights. We found the results to 
be robust to the particular weights used. However, there is no assurance that 
all possible weights would yield equivalent results for all possible analyses. 

• Bias associated with student exit from secondary school before sample 
selection. Because students were selected from the 1985-86 school year 
rosters, the student universe excluded special education students who had left 
school before that date and were removed from the rosters. The 13- to 15- 
year-old cohort is probably not severely affected by this exclusion. The 16- to 
18-year-old cohort probably excludes a number of the mildly handicapped, who 
graduated or otherwise left school. The 19-year-old and above cohort tends to 
contain the moderately to severely handicapped who are more likely to remain 
in secondary school to the legal age limit (often 21). Thus, data from this first 
wave presents a biased view of older special education students. Because 
there were fewer disabled students age 19 and above in the universe than in 
other age groups, this bias should be modest in analyses that include other age 
groups. This bias should be remedied in the 1990 data collection effort, when 
the 13- to 15-year-old cohort (which is essentially unaffected by this type of 
censoring) will be 18 to 20 years old, allowing older special education students 
to be more representatively characterized. 



^ 66 S3 

ERIC 



• Potential bias associated with weight truncation. Students in three low- 
incidence disability categories (deaf, orthopedically impaired, and visually 
impaired) were found infrequently in medium- and smaller-sized districts, but 
often enough that they would have been assigned extremely large weights if 
they had been allowed to represent fully the special education students with 
these disabilities in those districts. It was considered advisable to adjust these 
weights downward to increase the effective sample size. Weights of these 
students in the small and medium-size district strata were adjusted slightly 
downward, and corresponding increases were made to the weights of students 
in the large district stratum (under the assumption that students in large and 
sometimes very large districts would more closely resemble students in 
medium- and smaller-sized districts than students in huge districts). 
Responses before and atter these weighting adjustments were nearly identical 
(see Table 10). but there is no assurance that they would be identical for all 
possible analyses. !n addition, because there were only three deaf/blind 
respondents from medium- or smaller-sized districts, and these three 
respondents had large weights, they were discarded. Thus, NLTS results do 
not represent deaf/blind students in medium- or smaller-sized LEAs, whose 
numbers cannot be accurately estimated from the NLTS survey. 

• Item nonresponse. In addition to the absence of whole instruments from 
which an item was taken, respondents may have failed to provide answers for 
specific questions. Usually, means and percentages are calculated on the 
basis of the responses from those for whom the question was appropriate and 
who answered the question. There is no assurance that those who failed to 
answer an appropriate question (that is, who were led by the questionnaire skip 
logic to the question but who declined lo answer) are similar to those who 
answered the question. However, the extent of missing data can be assessed 
for each question. Item response rates for the NLTS data collection 
instruments were very high, with the exception of household income, IQ. and 
number of days absent from school. 



Overail Assessment of Data Usefulness 

The NLTS study is the largest and most comprehensive research effort ever 
conducted about secondary special education students in the United States, For it to 
fulfill its purpose of providing useful information to practitioners, policymakers, 
researchers, and others in the special education community regarding the transition of 
youth with disabilities from secondary school to early adulthood, the data must meet high 
standards of quality. As documented in this report, the NLTS has been subject to 
relatively high nonresponse rates. Therefore, it is necessary to address the issue of 
whether high nonresponse rates have compromised the usefulness of the data, and 
what types of analyses would be most appropriate in the presence of high nonresponse 
rates. 

To place the nonresponse issue in perspective, it is useful to consider (1) traditional 
survey research procedural standards. (2) traditional survey research response rates, 
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(3) the sources of nonresponse (4) the magnitudes of nonresponse biases, and (5) 
traditional studies in special education. 

(1 ) The procedures followed in the NLTS meet a very high standard for 
traditional survey research. Considerable care and skill went into research 
design and into extensive efforts to persuade unwilling districts to 
participate, locate and persuade parents to participate, locate school 
records, investigate biases, and calculate appropriate weights. Thus, high 
nonresponse rates were not a result of inadequate procedures. 

(2) The response rates for the NLTS were lower than those typically obtained in 
traditional sun/ey research, which are in the range of 70% to 85%. Much of 
the difference was attributable to special circumstances of the NLTS that are 
not present in traditional surveys, especially the need to sample LEAs and 
state-operated schools and obtain their participation, the reluctance of some 
LEAs to provide rosters, and the lack of telephones of many parents/ 
guardians. If we compare NLTS response rates with traditional random digit 
telephone surveys, after adjusting for these special circumstances, we find 
the response rates to be favorable. For example, out of the 12,833 students 
selected for the sample, 4,502 were nonrespondents for special 
circumstances not found in traditional random digit dialing telephone surveys 
(i.e.. 1 .632 youth whose names were not revealed by the school district 
because their parents did not return a consent form and therefore could not 
be called or located by mail, 636 youth who had inaccurate location 
information and whose parents could not be located by mail or telephone. 43 
youth who were deceased, 187 youth who had no adult who could respond 
for the youth, and 2,004 youth who had no working telephone number and 
whose parenVguardian would not call a toll-free number). Out of the 
remaining 8.331 sampled youth, complete or almost complete parenV 
guardian telephone surveys were obtained for 6,693 youth (and an 
additional 441 respondents were obtained via in-person surveys). Thus, on 
a comparable basis to random digit dialing surveys, the response rate was 
6,693 / 8,331 = 80.3%. Nevertheless, special circumstances resulted in 
overall response rates that were sufficiently low so that there always will be 
some question concerning the representativeness of survey results. 

(3) Although there were a variety of sources of nonresponse, the most severe 
was the unwillingness of LEAs and state-operated schools to participate or 
to release the names and addresses of parents before obtaining their written 
permission. Unwillingness to participate resulted in the loss of 51 .8% of the 
sampled LEAs and 73.8% of the sampled state-operated schools. 
Unwillingness to provide rosters to SRI resulted in the loss of 1 ,632 students 
out of an original sample of 12.833. In the absence of a legislative 
requirement or substantial financial incentive for LEA/school participation, it 
is unlikely that any .study of special education students would be abie to 
improve these nonresponse rates substantially. Other sources of 
nonresponse included inaccurate location information, inability to locate 
school records, and the presence of large numbers of households that 
lacked a working telephone. Decreasing nonresponse rates from inaccurate 
location information would require implementing a prospective survey 
design, and decreasing nonresponse rates from a lack of telephones would 
require conducting in-person interviews. These alternatives would have 
increased survey costs substantially. 
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^4) Our examination of the magnitude of response bias attributable to LEA 

nonparticipation, nonresponse by parents/guardians, inability to locate scliool 
records, and item nonresponse gives us reason to believe that these biases 
would not seriously jeopardize the representativeness of the NLTS database. 
The most serious potential problem was caused by nonresponse bias in the 
parent/guardian interview, and we believe that our income weighting 
adjustments have been successful in substantially reducing those biases. We 
are particularly encouraged by our ability to predict school status using the 
weighted parent/guardian and school record abstract data. However, there are 
no guarantees. 

(5) The NLTS far exceeds the scope of previous transition research studies in 
special education. Previous research focused on single states and/or disability 
categories, contained small samples, and were subject to the same types of 
response rates and nonrespondent errors. 

Despite the effort that has gone into quantifying and minimizing nonresponse bias (via a 
nonresponse survey and reweighting) and the encouraging results of these efforts, there will 
always be some uncertainty concerning NLTS representativeness relative to the entire 
universe of special education students. All available evidence suggests that the NLTS can 
yield valid results for a large portion of the special education student universe. For example, 
analyses conducted by SRI International on NLTS data (e.g., Wagner et al., 1990) have 
yielded point estimates and relationships among variables that are entirely consistent with 
other research with which we are familiar in special education and regarding the general 
population of youth. However, NLTS estimates of the number of special education students 
in some disability categories are not as consistent as we would have desired with counts in 
the 10th Annual Report to Congress. Also, it is not clear that all special education students 
are fully represented. Nonresponse biases appear to be related to lower participation rates 
by lower-income students and those who had left school before graduation. Therefore, 
there is a possibility that the NLTS results, while accurately portraying the experiences of the 
vast majority of special education students, may have failed to capture completely the 
experiences of a small minority of special education students. 

Even with these concerns, we believe that the NLTS database is more than merely 
adequate. It is both good and representative, and will prove useful and valid for policy 
decisions and for social and scientific analysis. The NLTS meets very high standards with 
respect to its procedures. Although its overall response rate was low because of special 
causes not present in traditional survey research, when measured on a basis comparable to 
traditional random digit dialing telephone research, response rates were very good. 
Furthermore, the NLTS far exceeds the scope of previous transition research studies in 
special education, and can provide insights not previously available from other sources. It 
also represents the best data that can be obtained without incurring extraordinary costs. 
Higher response rates in special education surveys of this type will not be obtained until 
special causes of nonresponse are addressed via legislative sanctions and/or incentives for 
LE A/school participation and disclosure of rosters. Prospective survey designs with in- 
person interviews also would be required. 
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